Max level shown:
12025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
22025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
32025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
42025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
52025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
72025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
82025-09-03T04:15:07.265ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-09-03 04:05:07.265549344 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
92025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
102025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
112025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
122025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
132025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
142025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
152025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
162025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
172025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
182025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
192025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
202025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
212025-09-03T04:15:07.265ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:133 reason = blueprint disabled target_id = 60e73e54-9659-486a-96ba-b69df28d8796
222025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 20.97\u{b5}s iteration = 13 status = {"enabled":false,"target_id":"60e73e54-9659-486a-96ba-b69df28d8796"}
232025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
242025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
252025-09-03T04:15:07.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
262025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
272025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
282025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
292025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
302025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
312025-09-03T04:15:07.266ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d788cd6-59e1-4eee-8401-73a64a0795db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("33692814-cb11-4468-bf51-4d6d8418d856.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1718, tv_nsec: 939846080 } })
322025-09-03T04:15:07.266ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2085efc3-987d-4d25-bf4e-dbfa977a0d96.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1718, tv_nsec: 939845831 } })
332025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
342025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
352025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.37762ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
362025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
372025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.277009ms iteration = 12 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
382025-09-03T04:15:07.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
392025-09-03T04:15:07.268ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
402025-09-03T04:15:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
412025-09-03T04:15:07.269ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 133 reason = Timeout
422025-09-03T04:15:07.269ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
432025-09-03T04:15:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
442025-09-03T04:15:07.271ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1718, tv_nsec: 943290245 } })
452025-09-03T04:15:07.271ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
462025-09-03T04:15:07.271ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-09-03 04:05:07.265549344 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
472025-09-03T04:15:07.271ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 6.103274ms iteration = 12 status = {"expiration":"2025-09-03T04:05:07.265549344Z","pruned":[]}
482025-09-03T04:15:07.272ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1718, tv_nsec: 945629602 } })
492025-09-03T04:15:07.272ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
502025-09-03T04:15:07.272ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
512025-09-03T04:15:07.272ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
522025-09-03T04:15:07.272ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.848449ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
532025-09-03T04:15:07.272ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
542025-09-03T04:15:07.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
552025-09-03T04:15:07.274ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Timeout
562025-09-03T04:15:07.274ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
572025-09-03T04:15:07.274ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.16\u{b5}s iteration = 15 status = "Disabled"
582025-09-03T04:15:07.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
592025-09-03T04:15:07.275ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
602025-09-03T04:15:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
612025-09-03T04:15:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
622025-09-03T04:15:07.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
632025-09-03T04:15:07.277ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
642025-09-03T04:15:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
652025-09-03T04:15:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
662025-09-03T04:15:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
672025-09-03T04:15:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
682025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
692025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
702025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
712025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
722025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
732025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 10.007952ms iteration = 133 status = {"chicken_switches_updated":false}
742025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
752025-09-03T04:15:07.279ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
762025-09-03T04:15:07.280ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
772025-09-03T04:15:07.280ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
782025-09-03T04:15:07.280ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
792025-09-03T04:15:07.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
802025-09-03T04:15:07.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
812025-09-03T04:15:07.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
822025-09-03T04:15:07.282ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
832025-09-03T04:15:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
842025-09-03T04:15:07.283ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
852025-09-03T04:15:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
862025-09-03T04:15:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
872025-09-03T04:15:07.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
882025-09-03T04:15:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
892025-09-03T04:15:07.285ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
902025-09-03T04:15:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
912025-09-03T04:15:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
922025-09-03T04:15:07.285ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
932025-09-03T04:15:07.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
942025-09-03T04:15:07.286ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
952025-09-03T04:15:07.286ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
962025-09-03T04:15:07.286ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
972025-09-03T04:15:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
982025-09-03T04:15:07.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
992025-09-03T04:15:07.287ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
1002025-09-03T04:15:07.287ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 0 version = 2
1012025-09-03T04:15:07.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1022025-09-03T04:15:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1032025-09-03T04:15:07.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1042025-09-03T04:15:07.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1052025-09-03T04:15:07.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
1062025-09-03T04:15:07.289ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1072025-09-03T04:15:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1082025-09-03T04:15:07.290ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
1092025-09-03T04:15:07.290ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
1102025-09-03T04:15:07.290ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
1112025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1122025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Timeout
1132025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1142025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1152025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Timeout
1162025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1172025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
1182025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1192025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1202025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1212025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1222025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1232025-09-03T04:15:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1242025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
1252025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1262025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
1272025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1282025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1292025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
1302025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
1312025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1322025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1332025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
1342025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1352025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
1362025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
1372025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
1382025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1392025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
1402025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1412025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
1422025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1432025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
1442025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1452025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
1462025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1472025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
1482025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
1492025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1502025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
1512025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1522025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1532025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
1542025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1552025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
1562025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
1572025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Timeout
1582025-09-03T04:15:07.292ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1592025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
1602025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1612025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1622025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
1632025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1642025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 9.211\u{b5}s iteration = 23 status = {"disabled":true,"errors":[],"sps":[]}
1652025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
1662025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
1672025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1682025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1692025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
1702025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
1712025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1722025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
1732025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1742025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1752025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
1762025-09-03T04:15:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1772025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
1782025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
1792025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
1802025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1812025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
1822025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
1832025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1842025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1852025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
1862025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
1872025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
1882025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
1892025-09-03T04:15:07.293ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1902025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 4 version = 2
1912025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
1922025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-09-03 04:04:25.477141 UTC dns_group = external generation = 2
1932025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 29.05041ms iteration = 12 status = {"generation":2}
1942025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 2.398117ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
1952025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1962025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1972025-09-03T04:15:07.294ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.461958ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
1982025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
1992025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
2002025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-09-03 04:04:24.694355 UTC dns_group = internal generation = 1
2012025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 29.670455ms iteration = 12 status = {"generation":1}
2022025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2032025-09-03T04:15:07.295ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
2042025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.874491ms iteration = 12 status = {"by_rx":{},"error":null}
2052025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.944762ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
2062025-09-03T04:15:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2072025-09-03T04:15:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2082025-09-03T04:15:07.296ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2092025-09-03T04:15:07.296ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2102025-09-03T04:15:07.296ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 4.172191ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2112025-09-03T04:15:07.296ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2122025-09-03T04:15:07.296ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2132025-09-03T04:15:07.296ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2142025-09-03T04:15:07.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.756314ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
2152025-09-03T04:15:07.297ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
2162025-09-03T04:15:07.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2172025-09-03T04:15:07.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
2182025-09-03T04:15:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
2192025-09-03T04:15:07.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2202025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2212025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2222025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2232025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2242025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2252025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
2262025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
2272025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2282025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 7.059561ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"186.0.0","globs_reprocessed":{},"no_receivers":[]}
2292025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2302025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
2312025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
2322025-09-03T04:15:07.299ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2332025-09-03T04:15:07.300ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2342025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
2352025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2362025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2372025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2382025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2392025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2402025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
2412025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2422025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2432025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
2442025-09-03T04:15:07.300ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
2452025-09-03T04:15:07.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2462025-09-03T04:15:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2472025-09-03T04:15:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
2482025-09-03T04:15:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2492025-09-03T04:15:07.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2502025-09-03T04:15:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2512025-09-03T04:15:07.302ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2522025-09-03T04:15:07.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2532025-09-03T04:15:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2542025-09-03T04:15:07.302ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-09-03T04:04:05.025624Z, time_modified: 2025-09-03T04:04:05.025624Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }) }
2552025-09-03T04:15:07.302ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 37.213639ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]}},"default_endpoint":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
2562025-09-03T04:15:07.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2572025-09-03T04:15:07.303ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2582025-09-03T04:15:07.304ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1718, tv_nsec: 977650194 } })
2592025-09-03T04:15:07.304ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
2602025-09-03T04:15:07.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2612025-09-03T04:15:07.306ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2622025-09-03T04:15:07.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2632025-09-03T04:15:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
2642025-09-03T04:15:07.310ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
2652025-09-03T04:15:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
2662025-09-03T04:15:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
2672025-09-03T04:15:07.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
2682025-09-03T04:15:07.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
2692025-09-03T04:15:07.338ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
2702025-09-03T04:15:07.339ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 47.976107ms iteration = 67 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
2712025-09-03T04:15:07.370ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 77.40281ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
2722025-09-03T04:15:07.387ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2732025-09-03T04:15:07.387ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2742025-09-03T04:15:07.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2752025-09-03T04:15:07.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2762025-09-03T04:15:07.390ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2772025-09-03T04:15:07.391ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
2782025-09-03T04:15:07.391ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 100.010414ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2792025-09-03T04:15:07.391ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2802025-09-03T04:15:07.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
2812025-09-03T04:15:07.397ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
2822025-09-03T04:15:07.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2832025-09-03T04:15:07.397ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 105.376053ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2842025-09-03T04:15:07.401ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2852025-09-03T04:15:07.401ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
2862025-09-03T04:15:07.401ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 109.209571ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
2872025-09-03T04:15:07.403ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 111.271295ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
2882025-09-03T04:15:07.404ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 112.283053ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
2892025-09-03T04:15:07.417ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
2902025-09-03T04:15:07.417ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 124.748393ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
2912025-09-03T04:15:07.418ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2922025-09-03T04:15:07.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2932025-09-03T04:15:07.423ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 132.276828ms iteration = 25 status = {"physical_disks_added":0}
2942025-09-03T04:15:07.424ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2952025-09-03T04:15:07.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
2962025-09-03T04:15:07.432ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2972025-09-03T04:15:07.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2982025-09-03T04:15:07.445ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
2992025-09-03T04:15:07.445ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3002025-09-03T04:15:07.446ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1bb6fd0-08d3-4dcf-97d0-ed7ccc485e13", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3012025-09-03T04:15:07.446ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3022025-09-03T04:15:07.447ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3032025-09-03T04:15:07.448ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3042025-09-03T04:15:07.448ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
3052025-09-03T04:15:07.448ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
3062025-09-03T04:15:07.451ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3072025-09-03T04:15:07.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3082025-09-03T04:15:07.454ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3092025-09-03T04:15:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
3102025-09-03T04:15:07.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
3112025-09-03T04:15:07.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
3122025-09-03T04:15:07.462ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 171.521132ms iteration = 23 status = 0
3132025-09-03T04:15:07.473ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
3142025-09-03T04:15:07.494ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
3152025-09-03T04:15:07.494ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3162025-09-03T04:15:07.495ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "189b9cbd-7115-4927-9256-1942efe7f2d6", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3172025-09-03T04:15:07.495ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3182025-09-03T04:15:07.496ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3192025-09-03T04:15:07.512ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
3202025-09-03T04:15:07.512ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 221.209081ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3212025-09-03T04:15:07.544ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
3222025-09-03T04:15:07.544ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3232025-09-03T04:15:07.544ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b7fa910-1c4b-4c8e-8502-99f0921a7014", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3242025-09-03T04:15:07.544ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3252025-09-03T04:15:07.547ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3262025-09-03T04:15:07.558ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
3272025-09-03T04:15:07.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
3282025-09-03T04:15:07.569ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3292025-09-03T04:15:07.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3302025-09-03T04:15:07.633ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3312025-09-03T04:15:07.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3322025-09-03T04:15:07.635ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3332025-09-03T04:15:07.639ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3342025-09-03T04:15:07.639ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3352025-09-03T04:15:07.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3362025-09-03T04:15:07.641ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3372025-09-03T04:15:07.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
3382025-09-03T04:15:07.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
3392025-09-03T04:15:07.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
3402025-09-03T04:15:07.680ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
3412025-09-03T04:15:07.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
3422025-09-03T04:15:07.683ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 417.939706ms iteration = 23 status = null
3432025-09-03T04:15:07.699ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
3442025-09-03T04:15:07.699ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 406.815806ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
3452025-09-03T04:15:07.730ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
3462025-09-03T04:15:07.731ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "069c66a5-54f2-4e70-a638-c533bebe0951", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3472025-09-03T04:15:07.731ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3482025-09-03T04:15:07.733ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9944ed77-3062-4f5d-a964-a26afd043782", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3492025-09-03T04:15:07.733ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3502025-09-03T04:15:07.735ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
3512025-09-03T04:15:07.735ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 469.925142ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
3522025-09-03T04:15:07.766ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
3532025-09-03T04:15:07.808ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 542.76063ms iteration = 23 status = {}
3542025-09-03T04:15:07.811ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
3552025-09-03T04:15:07.811ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3562025-09-03T04:15:07.811ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae3a434c-b04e-411a-82fe-23f6684be094", "content-length": "372", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3572025-09-03T04:15:07.811ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3582025-09-03T04:15:07.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3592025-09-03T04:15:07.825ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
3602025-09-03T04:15:07.825ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
3612025-09-03T04:15:07.825ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ac6fc5c1-0fa8-4b67-a29f-d4f3f0e2f7be", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3622025-09-03T04:15:07.825ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3632025-09-03T04:15:07.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3642025-09-03T04:15:07.828ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3652025-09-03T04:15:07.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3662025-09-03T04:15:07.832ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3672025-09-03T04:15:07.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3682025-09-03T04:15:07.835ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
3692025-09-03T04:15:07.835ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
3702025-09-03T04:15:07.836ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "960a125f-9790-4a1e-99b1-643810ecd05a", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3712025-09-03T04:15:07.838ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 545.694152ms iteration = 23 status = {}
3722025-09-03T04:15:07.897ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
3732025-09-03T04:15:07.897ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "5dd077dd-cd52-4f05-94bd-8066b171bfc2", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3742025-09-03T04:15:07.897ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3752025-09-03T04:15:07.900ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "0de8fc08-c6d3-4974-9bdc-fa7d7e5496d0", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3762025-09-03T04:15:07.900ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3772025-09-03T04:15:07.902ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "b9d7c148-2bca-45a9-a9ac-2d880680728c", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3782025-09-03T04:15:07.902ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3792025-09-03T04:15:07.904ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
3802025-09-03T04:15:07.904ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 638.946497ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
3812025-09-03T04:15:07.922ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
3822025-09-03T04:15:07.923ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f5b3708-e5d1-4a7f-a41d-00893d6cb808", "content-length": "691", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
3832025-09-03T04:15:07.923ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
3842025-09-03T04:15:07.923ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
3852025-09-03T04:15:07.923ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
3862025-09-03T04:15:07.924ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 632.378518ms iteration = 23 status = {}
3872025-09-03T04:15:07.953ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3882025-09-03T04:15:07.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3892025-09-03T04:15:07.956ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
3902025-09-03T04:15:07.956ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3912025-09-03T04:15:07.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3922025-09-03T04:15:07.967ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
3932025-09-03T04:15:07.967ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
3942025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
3952025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
3962025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3972025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3982025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
3992025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4002025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
4012025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
4022025-09-03T04:15:07.968ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
4032025-09-03T04:15:07.968ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4042025-09-03T04:15:07.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4052025-09-03T04:15:07.971ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
4062025-09-03T04:15:07.971ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
4072025-09-03T04:15:08.194ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ea392c41-d083-4d0f-8525-7817ef16e36d", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
4082025-09-03T04:15:08.194ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
4092025-09-03T04:15:08.194ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
4102025-09-03T04:15:08.195ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d7f6fe0-e415-469d-9889-d82073af4084", "content-length": "682", "date": "Wed, 03 Sep 2025 04:15:07 GMT"} })
4112025-09-03T04:15:08.195ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4122025-09-03T04:15:08.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4132025-09-03T04:15:08.199ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4142025-09-03T04:15:08.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4152025-09-03T04:15:08.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4162025-09-03T04:15:08.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4172025-09-03T04:15:08.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
4182025-09-03T04:15:08.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
4192025-09-03T04:15:08.212ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 920.110422ms iteration = 23 status = {}
4202025-09-03T04:15:08.264ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
4212025-09-03T04:15:08.264ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::b]:12221
4222025-09-03T04:15:08.295ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/producers
4232025-09-03T04:15:08.295ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::b]:46418
4242025-09-03T04:15:08.295ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46418 req_id = 47dc2ce6-e64c-4b8f-a973-a23ebc00c2ea roles = RoleSet { roles: {} } uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
4252025-09-03T04:15:08.296ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46418 req_id = 47dc2ce6-e64c-4b8f-a973-a23ebc00c2ea resource = Database result = Ok(()) uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
4262025-09-03T04:15:08.345ZINFOnexus: assigned collector to new producer collector_id = 5ae9222c-aa77-4bb7-9fcb-c8d01163fee8 file = nexus/src/app/oximeter.rs:76 producer_id = fa992872-a69d-4760-b37f-1bd4fe4126ef
4272025-09-03T04:15:08.345ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 49413 local_addr = [fd00:1122:3344:101::b]:12221 method = POST remote_addr = [fd00:1122:3344:101::b]:46418 req_id = 47dc2ce6-e64c-4b8f-a973-a23ebc00c2ea response_code = 201 uri = /metrics/producers
4282025-09-03T04:15:08.345ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "47dc2ce6-e64c-4b8f-a973-a23ebc00c2ea", "content-length": "41", "date": "Wed, 03 Sep 2025 04:15:08 GMT"} })
4292025-09-03T04:15:08.346ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
4302025-09-03T04:15:09.652ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 333 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = fd9c182c-c52a-47d9-bbf5-cbbacc379b5b response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
4312025-09-03T04:15:12.268ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 134 reason = Timeout
4322025-09-03T04:15:12.268ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4332025-09-03T04:15:12.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4342025-09-03T04:15:12.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4352025-09-03T04:15:12.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4362025-09-03T04:15:12.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4372025-09-03T04:15:12.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4382025-09-03T04:15:12.275ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
4392025-09-03T04:15:12.275ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.114272ms iteration = 134 status = {"chicken_switches_updated":false}
4402025-09-03T04:15:17.269ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 135 reason = Timeout
4412025-09-03T04:15:17.269ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4422025-09-03T04:15:17.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4432025-09-03T04:15:17.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4442025-09-03T04:15:17.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4452025-09-03T04:15:17.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4462025-09-03T04:15:17.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4472025-09-03T04:15:17.277ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
4482025-09-03T04:15:17.277ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.378651ms iteration = 135 status = {"chicken_switches_updated":false}
4492025-09-03T04:15:17.290ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
4502025-09-03T04:15:17.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4512025-09-03T04:15:17.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4522025-09-03T04:15:17.292ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
4532025-09-03T04:15:17.292ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4542025-09-03T04:15:17.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4552025-09-03T04:15:17.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4562025-09-03T04:15:17.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4572025-09-03T04:15:17.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4582025-09-03T04:15:17.295ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
4592025-09-03T04:15:17.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4602025-09-03T04:15:17.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4612025-09-03T04:15:17.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4622025-09-03T04:15:17.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4632025-09-03T04:15:17.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4642025-09-03T04:15:17.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
4652025-09-03T04:15:17.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4662025-09-03T04:15:17.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4672025-09-03T04:15:17.316ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
4682025-09-03T04:15:17.316ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.722066ms iteration = 68 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
4692025-09-03T04:15:17.330ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 38.104986ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4702025-09-03T04:15:19.652ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 270 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 04e7ee41-a85b-4faf-bad8-50a7a047d82d response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
4712025-09-03T04:15:22.269ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 136 reason = Timeout
4722025-09-03T04:15:22.269ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4732025-09-03T04:15:22.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4742025-09-03T04:15:22.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4752025-09-03T04:15:22.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4762025-09-03T04:15:22.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4772025-09-03T04:15:22.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4782025-09-03T04:15:22.277ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
4792025-09-03T04:15:22.277ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.467551ms iteration = 136 status = {"chicken_switches_updated":false}
4802025-09-03T04:15:27.268ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 137 reason = Timeout
4812025-09-03T04:15:27.268ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4822025-09-03T04:15:27.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4832025-09-03T04:15:27.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4842025-09-03T04:15:27.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
4852025-09-03T04:15:27.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
4862025-09-03T04:15:27.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
4872025-09-03T04:15:27.275ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
4882025-09-03T04:15:27.275ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.98632ms iteration = 137 status = {"chicken_switches_updated":false}
4892025-09-03T04:15:27.290ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
4902025-09-03T04:15:27.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
4912025-09-03T04:15:27.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
4922025-09-03T04:15:27.292ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
4932025-09-03T04:15:27.292ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4942025-09-03T04:15:27.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4952025-09-03T04:15:27.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4962025-09-03T04:15:27.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
4972025-09-03T04:15:27.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
4982025-09-03T04:15:27.295ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.206393ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4992025-09-03T04:15:27.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
5002025-09-03T04:15:27.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5012025-09-03T04:15:27.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5022025-09-03T04:15:27.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5032025-09-03T04:15:27.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5042025-09-03T04:15:27.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5052025-09-03T04:15:27.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
5062025-09-03T04:15:27.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5072025-09-03T04:15:27.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5082025-09-03T04:15:27.316ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
5092025-09-03T04:15:27.316ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.741476ms iteration = 69 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
5102025-09-03T04:15:29.652ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 305 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = d1b738d3-6f0d-47d5-b3a9-bd2ebcf6616e response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
5112025-09-03T04:15:32.269ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 138 reason = Timeout
5122025-09-03T04:15:32.269ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5132025-09-03T04:15:32.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5142025-09-03T04:15:32.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5152025-09-03T04:15:32.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
5162025-09-03T04:15:32.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5172025-09-03T04:15:32.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5182025-09-03T04:15:32.277ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
5192025-09-03T04:15:32.277ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.563124ms iteration = 138 status = {"chicken_switches_updated":false}
5202025-09-03T04:15:37.266ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
5212025-09-03T04:15:37.266ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
5222025-09-03T04:15:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
5232025-09-03T04:15:37.266ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
5242025-09-03T04:15:37.266ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
5252025-09-03T04:15:37.266ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
5262025-09-03T04:15:37.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
5272025-09-03T04:15:37.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 139 reason = Timeout
5282025-09-03T04:15:37.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5292025-09-03T04:15:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5302025-09-03T04:15:37.271ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1748, tv_nsec: 943599692 } })
5312025-09-03T04:15:37.271ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1748, tv_nsec: 943685972 } })
5322025-09-03T04:15:37.271ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
5332025-09-03T04:15:37.271ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
5342025-09-03T04:15:37.272ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.396359ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
5352025-09-03T04:15:37.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5362025-09-03T04:15:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
5372025-09-03T04:15:37.278ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
5382025-09-03T04:15:37.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
5392025-09-03T04:15:37.281ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
5402025-09-03T04:15:37.281ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 11.336582ms iteration = 139 status = {"chicken_switches_updated":false}
5412025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
5422025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
5432025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
5442025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
5452025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
5462025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
5472025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
5482025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
5492025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
5502025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
5512025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
5522025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
5532025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
5542025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 13.14\u{b5}s iteration = 24 status = {"disabled":true,"errors":[],"sps":[]}
5552025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
5562025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
5572025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
5582025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
5592025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
5602025-09-03T04:15:37.292ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
5612025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
5622025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
5632025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
5642025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
5652025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
5662025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
5672025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
5682025-09-03T04:15:37.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
5692025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
5702025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
5712025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
5722025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
5732025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
5742025-09-03T04:15:37.294ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
5752025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
5762025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
5772025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
5782025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
5792025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
5802025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
5812025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5822025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
5832025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
5842025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
5852025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
5862025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
5872025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
5882025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
5892025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
5902025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
5912025-09-03T04:15:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
5922025-09-03T04:15:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
5932025-09-03T04:15:37.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5942025-09-03T04:15:37.298ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5952025-09-03T04:15:37.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5962025-09-03T04:15:37.299ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5972025-09-03T04:15:37.300ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
5982025-09-03T04:15:37.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
5992025-09-03T04:15:37.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6002025-09-03T04:15:37.301ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6012025-09-03T04:15:37.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
6022025-09-03T04:15:37.304ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6032025-09-03T04:15:37.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6042025-09-03T04:15:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6052025-09-03T04:15:37.305ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 11.922047ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
6062025-09-03T04:15:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6072025-09-03T04:15:37.305ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6082025-09-03T04:15:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6092025-09-03T04:15:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
6102025-09-03T04:15:37.306ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
6112025-09-03T04:15:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
6122025-09-03T04:15:37.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
6132025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6142025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6152025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6162025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6172025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
6182025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6192025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6202025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
6212025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
6222025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
6232025-09-03T04:15:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
6242025-09-03T04:15:37.310ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6252025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6262025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6272025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6282025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6292025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6302025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
6312025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6322025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
6332025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6342025-09-03T04:15:37.322ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1748, tv_nsec: 987928373 } })
6352025-09-03T04:15:37.322ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
6362025-09-03T04:15:37.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6372025-09-03T04:15:37.326ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
6382025-09-03T04:15:37.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6392025-09-03T04:15:37.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
6402025-09-03T04:15:37.328ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6412025-09-03T04:15:37.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
6422025-09-03T04:15:37.332ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6432025-09-03T04:15:37.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
6442025-09-03T04:15:37.348ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 55.2663ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
6452025-09-03T04:15:37.388ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
6462025-09-03T04:15:37.388ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 95.929894ms iteration = 70 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
6472025-09-03T04:15:37.393ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 100.227986ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
6482025-09-03T04:15:37.407ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6492025-09-03T04:15:37.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6502025-09-03T04:15:37.413ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
6512025-09-03T04:15:37.413ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6522025-09-03T04:15:37.413ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b8fb1f9-0c02-4c0f-84d3-6264fea8aefd", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
6532025-09-03T04:15:37.413ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6542025-09-03T04:15:37.416ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6552025-09-03T04:15:37.421ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
6562025-09-03T04:15:37.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
6572025-09-03T04:15:37.424ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
6582025-09-03T04:15:37.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
6592025-09-03T04:15:37.429ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 137.107433ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
6602025-09-03T04:15:37.432ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
6612025-09-03T04:15:37.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
6622025-09-03T04:15:37.436ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 141.662005ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
6632025-09-03T04:15:37.439ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 145.249031ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
6642025-09-03T04:15:37.442ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6652025-09-03T04:15:37.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6662025-09-03T04:15:37.443ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6672025-09-03T04:15:37.443ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
6682025-09-03T04:15:37.443ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6692025-09-03T04:15:37.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
6702025-09-03T04:15:37.446ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6712025-09-03T04:15:37.449ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
6722025-09-03T04:15:37.449ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
6732025-09-03T04:15:37.449ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
6742025-09-03T04:15:37.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
6752025-09-03T04:15:37.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
6762025-09-03T04:15:37.452ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6772025-09-03T04:15:37.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
6782025-09-03T04:15:37.458ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
6792025-09-03T04:15:37.458ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6802025-09-03T04:15:37.459ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bb6740b-b639-444b-94da-b081a14d7732", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
6812025-09-03T04:15:37.459ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6822025-09-03T04:15:37.459ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6832025-09-03T04:15:37.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6842025-09-03T04:15:37.460ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
6852025-09-03T04:15:37.460ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
6862025-09-03T04:15:37.461ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f8ddc22-8dea-4a1b-966d-4a3fe583ba8d", "content-length": "26", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
6872025-09-03T04:15:37.461ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
6882025-09-03T04:15:37.461ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
6892025-09-03T04:15:37.461ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
6902025-09-03T04:15:37.461ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
6912025-09-03T04:15:37.461ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "487eb465-7ff1-4c53-878a-2779f66649e7", "content-length": "525", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
6922025-09-03T04:15:37.461ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
6932025-09-03T04:15:37.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
6942025-09-03T04:15:37.462ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6952025-09-03T04:15:37.463ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
6962025-09-03T04:15:37.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
6972025-09-03T04:15:37.466ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
6982025-09-03T04:15:37.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
6992025-09-03T04:15:37.476ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
7002025-09-03T04:15:37.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
7012025-09-03T04:15:37.481ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 188.943258ms iteration = 24 status = 0
7022025-09-03T04:15:37.511ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 218.994736ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
7032025-09-03T04:15:37.522ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
7042025-09-03T04:15:37.522ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
7052025-09-03T04:15:37.522ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 229.871495ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
7062025-09-03T04:15:37.530ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
7072025-09-03T04:15:37.531ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 238.84305ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
7082025-09-03T04:15:37.558ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {37311ff8-5cde-4d05-aa40-7c01012a2b36: {10.151.1.175: {fadf70e0-83c4-4019-9200-6730d768ea9d}, 10.151.1.176: {fadf70e0-83c4-4019-9200-6730d768ea9d}}} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
7092025-09-03T04:15:37.558ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
7102025-09-03T04:15:37.558ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "231aa8de-fc14-4569-8938-a395e298b80c", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
7112025-09-03T04:15:37.558ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7122025-09-03T04:15:37.558ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7132025-09-03T04:15:37.562ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7142025-09-03T04:15:37.562ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7152025-09-03T04:15:37.566ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7162025-09-03T04:15:37.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7172025-09-03T04:15:37.569ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7182025-09-03T04:15:37.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7192025-09-03T04:15:37.570ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7202025-09-03T04:15:37.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7212025-09-03T04:15:37.574ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7222025-09-03T04:15:37.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7232025-09-03T04:15:37.578ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7242025-09-03T04:15:37.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7252025-09-03T04:15:37.579ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7262025-09-03T04:15:37.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7272025-09-03T04:15:37.581ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7282025-09-03T04:15:37.581ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 289.691948ms iteration = 26 status = {"physical_disks_added":0}
7292025-09-03T04:15:37.582ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
7302025-09-03T04:15:37.582ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7312025-09-03T04:15:37.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7322025-09-03T04:15:37.584ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7332025-09-03T04:15:37.584ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7342025-09-03T04:15:37.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7352025-09-03T04:15:37.586ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7362025-09-03T04:15:37.588ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7372025-09-03T04:15:37.588ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7382025-09-03T04:15:37.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7392025-09-03T04:15:37.592ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7402025-09-03T04:15:37.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7412025-09-03T04:15:37.594ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7422025-09-03T04:15:37.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7432025-09-03T04:15:37.596ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7442025-09-03T04:15:37.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7452025-09-03T04:15:37.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7462025-09-03T04:15:37.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7472025-09-03T04:15:37.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7482025-09-03T04:15:37.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7492025-09-03T04:15:37.603ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7502025-09-03T04:15:37.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7512025-09-03T04:15:37.605ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7522025-09-03T04:15:37.610ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = instance_watcher body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vmms/8f17b50d-9850-4af8-b7e4-8f461e962229/state
7532025-09-03T04:15:37.611ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: 317fce13-5ad5-4443-8e18-870782a5042a, lookup_type: ById(317fce13-5ad5-4443-8e18-870782a5042a) } result = Ok(())
7542025-09-03T04:15:37.611ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7552025-09-03T04:15:37.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7562025-09-03T04:15:37.618ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
7572025-09-03T04:15:37.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
7582025-09-03T04:15:37.621ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7592025-09-03T04:15:37.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7602025-09-03T04:15:37.622ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 356.785453ms iteration = 24 status = null
7612025-09-03T04:15:37.623ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7622025-09-03T04:15:37.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7632025-09-03T04:15:37.625ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7642025-09-03T04:15:37.630ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: 317fce13-5ad5-4443-8e18-870782a5042a, lookup_type: ById(317fce13-5ad5-4443-8e18-870782a5042a) } result = Ok(())
7652025-09-03T04:15:37.630ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7662025-09-03T04:15:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7672025-09-03T04:15:37.636ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7682025-09-03T04:15:37.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7692025-09-03T04:15:37.639ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7702025-09-03T04:15:37.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7712025-09-03T04:15:37.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7722025-09-03T04:15:37.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7732025-09-03T04:15:37.644ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7742025-09-03T04:15:37.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7752025-09-03T04:15:37.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7762025-09-03T04:15:37.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7772025-09-03T04:15:37.648ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7782025-09-03T04:15:37.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7792025-09-03T04:15:37.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7802025-09-03T04:15:37.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7812025-09-03T04:15:37.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7822025-09-03T04:15:37.660ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: fa799857-1c69-48bd-8804-4526a6b01654, lookup_type: ById(fa799857-1c69-48bd-8804-4526a6b01654) } result = Ok(())
7832025-09-03T04:15:37.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7842025-09-03T04:15:37.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7852025-09-03T04:15:37.666ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7862025-09-03T04:15:37.666ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 399.526583ms iteration = 24 status = {}
7872025-09-03T04:15:37.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7882025-09-03T04:15:37.668ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7892025-09-03T04:15:37.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7902025-09-03T04:15:37.670ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7912025-09-03T04:15:37.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7922025-09-03T04:15:37.674ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7932025-09-03T04:15:37.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7942025-09-03T04:15:37.676ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7952025-09-03T04:15:37.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7962025-09-03T04:15:37.678ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7972025-09-03T04:15:37.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7982025-09-03T04:15:37.679ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
7992025-09-03T04:15:37.680ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ad0f0d9-dac7-4d47-b7b5-ee65a130a894", "content-length": "811", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
8002025-09-03T04:15:37.680ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {VirtualNetworkInterfaceHost { physical_host_ip: fd00:1122:3344:101::1, virtual_ip: 172.30.0.5, virtual_mac: MacAddr(MacAddr6([168, 64, 37, 240, 0, 0])), vni: Vni(12030372) }} sled = buskin
8012025-09-03T04:15:37.680ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
8022025-09-03T04:15:37.680ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
8032025-09-03T04:15:37.680ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8042025-09-03T04:15:37.682ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 387.556926ms iteration = 24 status = {}
8052025-09-03T04:15:37.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: fa799857-1c69-48bd-8804-4526a6b01654, lookup_type: ByName("default") } result = Ok(())
8062025-09-03T04:15:37.686ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8072025-09-03T04:15:37.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8082025-09-03T04:15:37.688ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8092025-09-03T04:15:37.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8102025-09-03T04:15:37.690ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8112025-09-03T04:15:37.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8122025-09-03T04:15:37.692ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8132025-09-03T04:15:37.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8142025-09-03T04:15:37.698ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8152025-09-03T04:15:37.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8162025-09-03T04:15:37.700ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8172025-09-03T04:15:37.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8182025-09-03T04:15:37.703ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8192025-09-03T04:15:37.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8202025-09-03T04:15:37.705ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8212025-09-03T04:15:37.710ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = InternetGateway { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, lookup_type: ById(1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1) }, key: d9676708-e397-40de-a3cb-d7a767a4f75b, lookup_type: ById(d9676708-e397-40de-a3cb-d7a767a4f75b) }, key: da219404-a79b-4f5a-a91a-c7c949125ac4, lookup_type: ById(da219404-a79b-4f5a-a91a-c7c949125ac4) }, key: fadf70e0-83c4-4019-9200-6730d768ea9d, lookup_type: ByName("default") } result = Ok(())
8222025-09-03T04:15:37.722ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
8232025-09-03T04:15:37.724ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "7526b473-7a9e-4284-a16c-ea9c1fbd6d9f", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
8242025-09-03T04:15:37.727ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 433.949262ms iteration = 24 status = {}
8252025-09-03T04:15:37.729ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8262025-09-03T04:15:37.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8272025-09-03T04:15:37.732ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
8282025-09-03T04:15:37.732ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8292025-09-03T04:15:37.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8302025-09-03T04:15:37.750ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
8312025-09-03T04:15:37.750ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
8322025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
8332025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
8342025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8352025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8362025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8372025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8382025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
8392025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
8402025-09-03T04:15:37.751ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
8412025-09-03T04:15:37.751ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8422025-09-03T04:15:37.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8432025-09-03T04:15:37.755ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
8442025-09-03T04:15:37.755ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
8452025-09-03T04:15:38.005ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "131558dc-4f94-4fd5-845b-7b2ce8a757b1", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
8462025-09-03T04:15:38.005ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
8472025-09-03T04:15:38.005ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
8482025-09-03T04:15:38.013ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "55e10054-9de5-406a-816a-708e192d30e3", "content-length": "682", "date": "Wed, 03 Sep 2025 04:15:37 GMT"} })
8492025-09-03T04:15:38.013ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8502025-09-03T04:15:38.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8512025-09-03T04:15:38.018ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8522025-09-03T04:15:38.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8532025-09-03T04:15:38.024ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8542025-09-03T04:15:38.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8552025-09-03T04:15:38.027ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8562025-09-03T04:15:38.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8572025-09-03T04:15:38.038ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 744.657222ms iteration = 24 status = {}
8582025-09-03T04:15:39.653ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 416 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 77e0e145-a804-45a7-85ab-ad89538e29ac response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
8592025-09-03T04:15:42.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 140 reason = Timeout
8602025-09-03T04:15:42.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8612025-09-03T04:15:42.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8622025-09-03T04:15:42.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8632025-09-03T04:15:42.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
8642025-09-03T04:15:42.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8652025-09-03T04:15:42.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8662025-09-03T04:15:42.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
8672025-09-03T04:15:42.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.35732ms iteration = 140 status = {"chicken_switches_updated":false}
8682025-09-03T04:15:42.876ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = instance_watcher result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vmms/8f17b50d-9850-4af8-b7e4-8f461e962229/state", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba75ae95-7bb6-482d-af07-2487201f334b", "content-length": "131", "date": "Wed, 03 Sep 2025 04:15:42 GMT"} })
8692025-09-03T04:15:42.878ZDEBGnexus (BackgroundTasks): updating instance state background_task = instance_watcher instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 state = SledVmmState { vmm_state: VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-09-03T04:15:35.131592692Z }, migration_in: None, migration_out: None } vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229
8702025-09-03T04:15:42.879ZINFOnexus (BackgroundTasks): received new VMM runtime state from sled agent background_task = instance_watcher file = nexus/src/app/instance.rs:2140 instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb migration_state = Migrations { migration_in: None, migration_out: None } propolis_id = 8f17b50d-9850-4af8-b7e4-8f461e962229 sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229 vmm_state = VmmRuntimeState { state: Starting, gen: Generation(2), time_updated: 2025-09-03T04:15:35.131592692Z }
8712025-09-03T04:15:42.879ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb roles = RoleSet { roles: {} } sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229
8722025-09-03T04:15:42.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher instance_id = 93025ecf-bf8a-46be-81c6-2fe0202411bb resource = Database result = Ok(()) sled_id = 6ba5229c-c111-45f6-88aa-7f00b514c681 vmm_id = 8f17b50d-9850-4af8-b7e4-8f461e962229
8732025-09-03T04:15:42.939ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 1 total_failed = 0 total_incomplete = 0 total_instances = 1 update_sagas_queued = 0
8742025-09-03T04:15:42.940ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 5.647397872s iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{"starting":1},"pruned_instances":0,"total_instances":1,"update_sagas_queued":0}
8752025-09-03T04:15:47.269ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 141 reason = Timeout
8762025-09-03T04:15:47.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8772025-09-03T04:15:47.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8782025-09-03T04:15:47.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8792025-09-03T04:15:47.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
8802025-09-03T04:15:47.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
8812025-09-03T04:15:47.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
8822025-09-03T04:15:47.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
8832025-09-03T04:15:47.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.36337ms iteration = 141 status = {"chicken_switches_updated":false}
8842025-09-03T04:15:47.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
8852025-09-03T04:15:47.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8862025-09-03T04:15:47.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
8872025-09-03T04:15:47.293ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
8882025-09-03T04:15:47.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8892025-09-03T04:15:47.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8902025-09-03T04:15:47.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8912025-09-03T04:15:47.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
8922025-09-03T04:15:47.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
8932025-09-03T04:15:47.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.2297ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
8942025-09-03T04:15:47.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8952025-09-03T04:15:47.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8962025-09-03T04:15:47.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
8972025-09-03T04:15:47.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8982025-09-03T04:15:47.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
8992025-09-03T04:15:47.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9002025-09-03T04:15:47.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
9012025-09-03T04:15:47.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9022025-09-03T04:15:47.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9032025-09-03T04:15:47.319ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
9042025-09-03T04:15:47.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.521589ms iteration = 71 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
9052025-09-03T04:15:49.656ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3011 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 7694426e-c477-4c6d-bb41-ae733cd25004 response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
9062025-09-03T04:15:52.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 142 reason = Timeout
9072025-09-03T04:15:52.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9082025-09-03T04:15:52.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9092025-09-03T04:15:52.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9102025-09-03T04:15:52.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
9112025-09-03T04:15:52.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9122025-09-03T04:15:52.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9132025-09-03T04:15:52.277ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9142025-09-03T04:15:52.277ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.266693ms iteration = 142 status = {"chicken_switches_updated":false}
9152025-09-03T04:15:57.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 143 reason = Timeout
9162025-09-03T04:15:57.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9172025-09-03T04:15:57.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9182025-09-03T04:15:57.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9192025-09-03T04:15:57.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
9202025-09-03T04:15:57.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9212025-09-03T04:15:57.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9222025-09-03T04:15:57.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9232025-09-03T04:15:57.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.769086ms iteration = 143 status = {"chicken_switches_updated":false}
9242025-09-03T04:15:57.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
9252025-09-03T04:15:57.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9262025-09-03T04:15:57.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9272025-09-03T04:15:57.293ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
9282025-09-03T04:15:57.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9292025-09-03T04:15:57.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9302025-09-03T04:15:57.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9312025-09-03T04:15:57.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9322025-09-03T04:15:57.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9332025-09-03T04:15:57.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.18172ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
9342025-09-03T04:15:57.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
9352025-09-03T04:15:57.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9362025-09-03T04:15:57.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9372025-09-03T04:15:57.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9382025-09-03T04:15:57.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9392025-09-03T04:15:57.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9402025-09-03T04:15:57.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
9412025-09-03T04:15:57.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
9422025-09-03T04:15:57.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
9432025-09-03T04:15:57.318ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
9442025-09-03T04:15:57.318ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.469865ms iteration = 72 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
9452025-09-03T04:15:59.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 494 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 717071c9-820f-4f93-aacf-79edf31f025b response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
9462025-09-03T04:16:02.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 144 reason = Timeout
9472025-09-03T04:16:02.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9482025-09-03T04:16:02.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9492025-09-03T04:16:02.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9502025-09-03T04:16:02.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
9512025-09-03T04:16:02.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
9522025-09-03T04:16:02.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
9532025-09-03T04:16:02.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
9542025-09-03T04:16:02.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.935598ms iteration = 144 status = {"chicken_switches_updated":false}
9552025-09-03T04:16:04.647ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:1025 local_addr = [fd00:1122:3344:101::b]:12221 remote_addr = [fd00:1122:3344:101::d]:61432
9562025-09-03T04:16:04.647ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 16d41c96-27fd-4925-b771-3ab4046f3014 roles = RoleSet { roles: {} } uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
9572025-09-03T04:16:04.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 16d41c96-27fd-4925-b771-3ab4046f3014 resource = Database result = Ok(()) uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
9582025-09-03T04:16:04.649ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2682 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 16d41c96-27fd-4925-b771-3ab4046f3014 response_code = 200 uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending
9592025-09-03T04:16:04.650ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 4e8f85f1-3ef7-422a-8298-690b5acdd3b4 roles = RoleSet { roles: {} } uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
9602025-09-03T04:16:04.650ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 4e8f85f1-3ef7-422a-8298-690b5acdd3b4 resource = Database result = Ok(()) uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
9612025-09-03T04:16:04.652ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2771 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 4e8f85f1-3ef7-422a-8298-690b5acdd3b4 response_code = 200 uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D
9622025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
9632025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
9642025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
9652025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9662025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
9672025-09-03T04:16:07.266ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-09-03 04:06:07.266535084 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
9682025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
9692025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
9702025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
9712025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
9722025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
9732025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
9742025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
9752025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
9762025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
9772025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
9782025-09-03T04:16:07.266ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
9792025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
9802025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
9812025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
9822025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
9832025-09-03T04:16:07.267ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2085efc3-987d-4d25-bf4e-dbfa977a0d96.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1778, tv_nsec: 938958881 } })
9842025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
9852025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
9862025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
9872025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
9882025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
9892025-09-03T04:16:07.267ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:133 reason = blueprint disabled target_id = 60e73e54-9659-486a-96ba-b69df28d8796
9902025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
9912025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 18.78\u{b5}s iteration = 14 status = {"enabled":false,"target_id":"60e73e54-9659-486a-96ba-b69df28d8796"}
9922025-09-03T04:16:07.267ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d788cd6-59e1-4eee-8401-73a64a0795db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("33692814-cb11-4468-bf51-4d6d8418d856.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1778, tv_nsec: 939068472 } })
9932025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
9942025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
9952025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.253739ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
9962025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
9972025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.184119ms iteration = 13 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
9982025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
9992025-09-03T04:16:07.267ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
10002025-09-03T04:16:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10012025-09-03T04:16:07.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
10022025-09-03T04:16:07.269ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1778, tv_nsec: 940764394 } })
10032025-09-03T04:16:07.269ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
10042025-09-03T04:16:07.269ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10052025-09-03T04:16:07.269ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-09-03 04:06:07.266535084 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
10062025-09-03T04:16:07.269ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.079963ms iteration = 13 status = {"expiration":"2025-09-03T04:06:07.266535084Z","pruned":[]}
10072025-09-03T04:16:07.269ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10082025-09-03T04:16:07.270ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10092025-09-03T04:16:07.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 145 reason = Timeout
10102025-09-03T04:16:07.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10112025-09-03T04:16:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10122025-09-03T04:16:07.270ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1778, tv_nsec: 942514167 } })
10132025-09-03T04:16:07.271ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
10142025-09-03T04:16:07.271ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.388361ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
10152025-09-03T04:16:07.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10162025-09-03T04:16:07.272ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10172025-09-03T04:16:07.272ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10182025-09-03T04:16:07.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
10192025-09-03T04:16:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10202025-09-03T04:16:07.274ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10212025-09-03T04:16:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
10222025-09-03T04:16:07.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10232025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
10242025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
10252025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
10262025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10272025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
10282025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
10292025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10302025-09-03T04:16:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
10312025-09-03T04:16:07.276ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10322025-09-03T04:16:07.276ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Timeout
10332025-09-03T04:16:07.276ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
10342025-09-03T04:16:07.276ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 9.91\u{b5}s iteration = 16 status = "Disabled"
10352025-09-03T04:16:07.277ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
10362025-09-03T04:16:07.277ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.964851ms iteration = 145 status = {"chicken_switches_updated":false}
10372025-09-03T04:16:07.277ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10382025-09-03T04:16:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10392025-09-03T04:16:07.281ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10402025-09-03T04:16:07.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
10412025-09-03T04:16:07.283ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10422025-09-03T04:16:07.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
10432025-09-03T04:16:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
10442025-09-03T04:16:07.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10452025-09-03T04:16:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10462025-09-03T04:16:07.285ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10472025-09-03T04:16:07.288ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
10482025-09-03T04:16:07.288ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
10492025-09-03T04:16:07.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
10502025-09-03T04:16:07.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
10512025-09-03T04:16:07.291ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
10522025-09-03T04:16:07.291ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10532025-09-03T04:16:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
10542025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
10552025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
10562025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10572025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10582025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
10592025-09-03T04:16:07.292ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
10602025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
10612025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
10622025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
10632025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
10642025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
10652025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
10662025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10672025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10682025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
10692025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10702025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10712025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
10722025-09-03T04:16:07.292ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10732025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
10742025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10752025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10762025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10772025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10782025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10792025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10802025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
10812025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
10822025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
10832025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
10842025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
10852025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10862025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
10872025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
10882025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
10892025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
10902025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
10912025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
10922025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
10932025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
10942025-09-03T04:16:07.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
10952025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
10962025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
10972025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
10982025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10992025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
11002025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
11012025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
11022025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
11032025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
11042025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
11052025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
11062025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
11072025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
11082025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
11092025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
11102025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
11112025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
11122025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
11132025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
11142025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 10.93\u{b5}s iteration = 25 status = {"disabled":true,"errors":[],"sps":[]}
11152025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
11162025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
11172025-09-03T04:16:07.294ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
11182025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
11192025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11202025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
11212025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11222025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11232025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
11242025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
11252025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
11262025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
11272025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
11282025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
11292025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11302025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
11312025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
11322025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
11332025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
11342025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
11352025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
11362025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11372025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
11382025-09-03T04:16:07.294ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11392025-09-03T04:16:07.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11402025-09-03T04:16:07.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11412025-09-03T04:16:07.295ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11422025-09-03T04:16:07.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11432025-09-03T04:16:07.296ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11442025-09-03T04:16:07.296ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11452025-09-03T04:16:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11462025-09-03T04:16:07.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 3.264593ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
11472025-09-03T04:16:07.297ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
11482025-09-03T04:16:07.297ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 3.224534ms iteration = 13 status = {"by_rx":{},"error":null}
11492025-09-03T04:16:07.297ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11502025-09-03T04:16:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
11512025-09-03T04:16:07.298ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
11522025-09-03T04:16:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
11532025-09-03T04:16:07.299ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
11542025-09-03T04:16:07.299ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 5.799342ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
11552025-09-03T04:16:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
11562025-09-03T04:16:07.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11572025-09-03T04:16:07.300ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11582025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
11592025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
11602025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11612025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
11622025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
11632025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.796107ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
11642025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
11652025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
11662025-09-03T04:16:07.301ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11672025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11682025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
11692025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
11702025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
11712025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11722025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11732025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11742025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
11752025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11762025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
11772025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
11782025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11792025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-09-03 04:04:24.694355 UTC dns_group = internal generation = 1
11802025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 36.208532ms iteration = 13 status = {"generation":1}
11812025-09-03T04:16:07.302ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11822025-09-03T04:16:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
11832025-09-03T04:16:07.303ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
11842025-09-03T04:16:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
11852025-09-03T04:16:07.303ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11862025-09-03T04:16:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11872025-09-03T04:16:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
11882025-09-03T04:16:07.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
11892025-09-03T04:16:07.304ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 10.100013ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"186.0.0","globs_reprocessed":{},"no_receivers":[]}
11902025-09-03T04:16:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
11912025-09-03T04:16:07.304ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
11922025-09-03T04:16:07.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
11932025-09-03T04:16:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
11942025-09-03T04:16:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
11952025-09-03T04:16:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
11962025-09-03T04:16:07.305ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
11972025-09-03T04:16:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
11982025-09-03T04:16:07.305ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11992025-09-03T04:16:07.306ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1778, tv_nsec: 978129364 } })
12002025-09-03T04:16:07.306ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
12012025-09-03T04:16:07.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
12022025-09-03T04:16:07.307ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
12032025-09-03T04:16:07.307ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12042025-09-03T04:16:07.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
12052025-09-03T04:16:07.307ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12062025-09-03T04:16:07.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12072025-09-03T04:16:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12082025-09-03T04:16:07.309ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12092025-09-03T04:16:07.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
12102025-09-03T04:16:07.311ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
12112025-09-03T04:16:07.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
12122025-09-03T04:16:07.311ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
12132025-09-03T04:16:07.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
12142025-09-03T04:16:07.311ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12152025-09-03T04:16:07.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
12162025-09-03T04:16:07.312ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 1 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
12172025-09-03T04:16:07.312ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.06018ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":1,"total_instances":0,"update_sagas_queued":0}
12182025-09-03T04:16:07.313ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
12192025-09-03T04:16:07.313ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
12202025-09-03T04:16:07.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
12212025-09-03T04:16:07.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12222025-09-03T04:16:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
12232025-09-03T04:16:07.314ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
12242025-09-03T04:16:07.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
12252025-09-03T04:16:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
12262025-09-03T04:16:07.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
12272025-09-03T04:16:07.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
12282025-09-03T04:16:07.317ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 25.202453ms iteration = 6 status = {"nsuccess":5}
12292025-09-03T04:16:07.323ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 30.883563ms iteration = 27 status = {"physical_disks_added":0}
12302025-09-03T04:16:07.332ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
12312025-09-03T04:16:07.332ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
12322025-09-03T04:16:07.332ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
12332025-09-03T04:16:07.332ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12342025-09-03T04:16:07.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12352025-09-03T04:16:07.334ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12362025-09-03T04:16:07.341ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12372025-09-03T04:16:07.341ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12382025-09-03T04:16:07.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12392025-09-03T04:16:07.344ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
12402025-09-03T04:16:07.344ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 51.940486ms iteration = 73 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
12412025-09-03T04:16:07.347ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
12422025-09-03T04:16:07.347ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 0 version = 2
12432025-09-03T04:16:07.347ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12442025-09-03T04:16:07.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12452025-09-03T04:16:07.350ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12462025-09-03T04:16:07.352ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-09-03T04:04:05.025624Z, time_modified: 2025-09-03T04:04:05.025624Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
12472025-09-03T04:16:07.352ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 85.982492ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]}},"default_endpoint":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
12482025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
12492025-09-03T04:16:07.353ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
12502025-09-03T04:16:07.353ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 59.306739ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
12512025-09-03T04:16:07.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
12522025-09-03T04:16:07.356ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
12532025-09-03T04:16:07.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
12542025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 4 version = 2
12552025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
12562025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-09-03 04:04:25.477141 UTC dns_group = external generation = 2
12572025-09-03T04:16:07.364ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 97.890788ms iteration = 13 status = {"generation":2}
12582025-09-03T04:16:07.367ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
12592025-09-03T04:16:07.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
12602025-09-03T04:16:07.376ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.317146ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
12612025-09-03T04:16:07.382ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12622025-09-03T04:16:07.382ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
12632025-09-03T04:16:07.382ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
12642025-09-03T04:16:07.383ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 90.427275ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
12652025-09-03T04:16:07.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12662025-09-03T04:16:07.384ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 90.958708ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
12672025-09-03T04:16:07.391ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12682025-09-03T04:16:07.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12692025-09-03T04:16:07.395ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 101.408924ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
12702025-09-03T04:16:07.397ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12712025-09-03T04:16:07.397ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
12722025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
12732025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
12742025-09-03T04:16:07.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
12752025-09-03T04:16:07.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 107.560359ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12762025-09-03T04:16:07.403ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
12772025-09-03T04:16:07.403ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
12782025-09-03T04:16:07.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
12792025-09-03T04:16:07.404ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 110.52235ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
12802025-09-03T04:16:07.407ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.025763ms iteration = 25 status = 0
12812025-09-03T04:16:07.409ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
12822025-09-03T04:16:07.409ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 116.620584ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
12832025-09-03T04:16:07.410ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12842025-09-03T04:16:07.412ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
12852025-09-03T04:16:07.412ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
12862025-09-03T04:16:07.412ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
12872025-09-03T04:16:07.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
12882025-09-03T04:16:07.413ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12892025-09-03T04:16:07.413ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12902025-09-03T04:16:07.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12912025-09-03T04:16:07.422ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12922025-09-03T04:16:07.425ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12932025-09-03T04:16:07.425ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12942025-09-03T04:16:07.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12952025-09-03T04:16:07.427ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
12962025-09-03T04:16:07.427ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
12972025-09-03T04:16:07.428ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "09186076-a721-4ec2-9101-2a66702a75b1", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
12982025-09-03T04:16:07.428ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
12992025-09-03T04:16:07.428ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
13002025-09-03T04:16:07.429ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13012025-09-03T04:16:07.429ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
13022025-09-03T04:16:07.429ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
13032025-09-03T04:16:07.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
13042025-09-03T04:16:07.431ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13052025-09-03T04:16:07.509ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
13062025-09-03T04:16:07.509ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13072025-09-03T04:16:07.510ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "823b501e-a4f5-4718-a66a-0c16000d8cae", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13082025-09-03T04:16:07.510ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13092025-09-03T04:16:07.511ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
13102025-09-03T04:16:07.516ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
13112025-09-03T04:16:07.516ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
13122025-09-03T04:16:07.516ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 222.229339ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
13132025-09-03T04:16:07.517ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13142025-09-03T04:16:07.519ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
13152025-09-03T04:16:07.519ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
13162025-09-03T04:16:07.519ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
13172025-09-03T04:16:07.519ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5fc41d2-ecf5-4edf-aafe-eae008df68bd", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13182025-09-03T04:16:07.519ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
13192025-09-03T04:16:07.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
13202025-09-03T04:16:07.523ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
13212025-09-03T04:16:07.553ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13222025-09-03T04:16:07.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13232025-09-03T04:16:07.561ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13242025-09-03T04:16:07.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13252025-09-03T04:16:07.563ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13262025-09-03T04:16:07.566ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13272025-09-03T04:16:07.566ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13282025-09-03T04:16:07.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13292025-09-03T04:16:07.568ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13302025-09-03T04:16:07.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
13312025-09-03T04:16:07.570ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
13322025-09-03T04:16:07.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
13332025-09-03T04:16:07.593ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
13342025-09-03T04:16:07.593ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13352025-09-03T04:16:07.594ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6aa36f9-0558-4845-84a7-dcc74666d52a", "content-length": "372", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13362025-09-03T04:16:07.594ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13372025-09-03T04:16:07.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13382025-09-03T04:16:07.614ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
13392025-09-03T04:16:07.614ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13402025-09-03T04:16:07.614ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7c29ecac-5174-412f-82ef-fc0557196a1a", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13412025-09-03T04:16:07.614ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13422025-09-03T04:16:07.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13432025-09-03T04:16:07.617ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13442025-09-03T04:16:07.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13452025-09-03T04:16:07.621ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13462025-09-03T04:16:07.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13472025-09-03T04:16:07.621ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
13482025-09-03T04:16:07.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
13492025-09-03T04:16:07.623ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 357.162266ms iteration = 25 status = null
13502025-09-03T04:16:07.624ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13512025-09-03T04:16:07.624ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13522025-09-03T04:16:07.624ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "09df9417-3b25-4589-b221-463c7708eb6c", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13532025-09-03T04:16:07.626ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 332.156534ms iteration = 25 status = {}
13542025-09-03T04:16:07.653ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
13552025-09-03T04:16:07.654ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "f072a596-4ef7-4d85-be2c-a14f02707d91", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13562025-09-03T04:16:07.654ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13572025-09-03T04:16:07.656ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "be7bab2a-f78b-4b70-84e0-c9c4c99455ab", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13582025-09-03T04:16:07.656ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13592025-09-03T04:16:07.658ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
13602025-09-03T04:16:07.658ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 391.524045ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
13612025-09-03T04:16:07.682ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
13622025-09-03T04:16:07.682ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 388.18828ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
13632025-09-03T04:16:07.697ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
13642025-09-03T04:16:07.753ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
13652025-09-03T04:16:07.754ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8c79eb8-8de2-427e-8894-8e6ee73aaf76", "content-length": "691", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
13662025-09-03T04:16:07.754ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
13672025-09-03T04:16:07.754ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
13682025-09-03T04:16:07.754ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
13692025-09-03T04:16:07.755ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 461.699232ms iteration = 25 status = {}
13702025-09-03T04:16:07.786ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 518.757535ms iteration = 25 status = {}
13712025-09-03T04:16:07.802ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
13722025-09-03T04:16:07.803ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1320d62a-658d-4f08-bf0b-9f7f5e2565d0", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13732025-09-03T04:16:07.803ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13742025-09-03T04:16:07.805ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "b1e9bfe5-db2f-4cb7-9a0c-3332ded64b29", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13752025-09-03T04:16:07.805ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13762025-09-03T04:16:07.807ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "84389d1c-b741-4ba6-b887-6f09c94434bf", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13772025-09-03T04:16:07.807ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13782025-09-03T04:16:07.808ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
13792025-09-03T04:16:07.808ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 542.088154ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13802025-09-03T04:16:07.852ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13812025-09-03T04:16:07.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13822025-09-03T04:16:07.855ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
13832025-09-03T04:16:07.855ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13842025-09-03T04:16:07.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13852025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
13862025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
13872025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
13882025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13892025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13902025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13912025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13922025-09-03T04:16:07.868ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13932025-09-03T04:16:07.869ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13942025-09-03T04:16:07.869ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
13952025-09-03T04:16:07.869ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
13962025-09-03T04:16:07.869ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13972025-09-03T04:16:07.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13982025-09-03T04:16:07.872ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
13992025-09-03T04:16:07.872ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14002025-09-03T04:16:07.989ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "400bba87-ab42-48c2-b477-98a6a1e22e5c", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
14012025-09-03T04:16:07.989ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
14022025-09-03T04:16:07.989ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
14032025-09-03T04:16:07.990ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6034e34d-67ae-4710-9798-443c0869d951", "content-length": "682", "date": "Wed, 03 Sep 2025 04:16:07 GMT"} })
14042025-09-03T04:16:07.990ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14052025-09-03T04:16:07.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14062025-09-03T04:16:07.992ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14072025-09-03T04:16:07.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14082025-09-03T04:16:07.996ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14092025-09-03T04:16:07.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14102025-09-03T04:16:07.999ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14112025-09-03T04:16:07.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14122025-09-03T04:16:08.006ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 712.79728ms iteration = 25 status = {}
14132025-09-03T04:16:09.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 301 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 65619270-fbae-4467-8eec-5fcd23aff395 response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
14142025-09-03T04:16:12.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 146 reason = Timeout
14152025-09-03T04:16:12.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14162025-09-03T04:16:12.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14172025-09-03T04:16:12.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14182025-09-03T04:16:12.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14192025-09-03T04:16:12.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14202025-09-03T04:16:12.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14212025-09-03T04:16:12.277ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14222025-09-03T04:16:12.277ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.191182ms iteration = 146 status = {"chicken_switches_updated":false}
14232025-09-03T04:16:17.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 147 reason = Timeout
14242025-09-03T04:16:17.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14252025-09-03T04:16:17.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14262025-09-03T04:16:17.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272025-09-03T04:16:17.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14282025-09-03T04:16:17.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14292025-09-03T04:16:17.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14302025-09-03T04:16:17.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14312025-09-03T04:16:17.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.561414ms iteration = 147 status = {"chicken_switches_updated":false}
14322025-09-03T04:16:17.292ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
14332025-09-03T04:16:17.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14342025-09-03T04:16:17.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14352025-09-03T04:16:17.293ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
14362025-09-03T04:16:17.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14372025-09-03T04:16:17.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14382025-09-03T04:16:17.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14392025-09-03T04:16:17.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14402025-09-03T04:16:17.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14412025-09-03T04:16:17.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.02549ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14422025-09-03T04:16:17.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14432025-09-03T04:16:17.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14442025-09-03T04:16:17.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14452025-09-03T04:16:17.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14462025-09-03T04:16:17.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14472025-09-03T04:16:17.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14482025-09-03T04:16:17.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
14492025-09-03T04:16:17.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14502025-09-03T04:16:17.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14512025-09-03T04:16:17.320ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
14522025-09-03T04:16:17.320ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.828492ms iteration = 74 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
14532025-09-03T04:16:19.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 281 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = c83a5814-93ea-4932-aca6-9ea59ebd42e0 response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
14542025-09-03T04:16:22.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 148 reason = Timeout
14552025-09-03T04:16:22.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14562025-09-03T04:16:22.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14572025-09-03T04:16:22.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14582025-09-03T04:16:22.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14592025-09-03T04:16:22.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14602025-09-03T04:16:22.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14612025-09-03T04:16:22.279ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14622025-09-03T04:16:22.279ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.874107ms iteration = 148 status = {"chicken_switches_updated":false}
14632025-09-03T04:16:27.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 149 reason = Timeout
14642025-09-03T04:16:27.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14652025-09-03T04:16:27.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14662025-09-03T04:16:27.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14672025-09-03T04:16:27.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14682025-09-03T04:16:27.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14692025-09-03T04:16:27.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14702025-09-03T04:16:27.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
14712025-09-03T04:16:27.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.990361ms iteration = 149 status = {"chicken_switches_updated":false}
14722025-09-03T04:16:27.293ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
14732025-09-03T04:16:27.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14742025-09-03T04:16:27.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14752025-09-03T04:16:27.293ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
14762025-09-03T04:16:27.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14772025-09-03T04:16:27.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14782025-09-03T04:16:27.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14792025-09-03T04:16:27.296ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
14802025-09-03T04:16:27.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
14812025-09-03T04:16:27.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
14822025-09-03T04:16:27.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14832025-09-03T04:16:27.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14842025-09-03T04:16:27.298ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 5.079826ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
14852025-09-03T04:16:27.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14862025-09-03T04:16:27.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14872025-09-03T04:16:27.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14882025-09-03T04:16:27.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
14892025-09-03T04:16:27.304ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
14902025-09-03T04:16:27.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
14912025-09-03T04:16:27.316ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
14922025-09-03T04:16:27.316ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.4731ms iteration = 75 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
14932025-09-03T04:16:29.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 477 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = ceae10e2-d79e-440f-9499-8481a74fff71 response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
14942025-09-03T04:16:32.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 150 reason = Timeout
14952025-09-03T04:16:32.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
14962025-09-03T04:16:32.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
14972025-09-03T04:16:32.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14982025-09-03T04:16:32.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
14992025-09-03T04:16:32.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
15002025-09-03T04:16:32.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
15012025-09-03T04:16:32.279ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
15022025-09-03T04:16:32.279ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.411341ms iteration = 150 status = {"chicken_switches_updated":false}
15032025-09-03T04:16:34.647ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 8c6f7115-d85e-40fb-b577-9bc6fe8bd8f4 roles = RoleSet { roles: {} } uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
15042025-09-03T04:16:34.647ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 8c6f7115-d85e-40fb-b577-9bc6fe8bd8f4 resource = Database result = Ok(()) uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
15052025-09-03T04:16:34.650ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 3421 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 8c6f7115-d85e-40fb-b577-9bc6fe8bd8f4 response_code = 200 uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?sort_by=id_ascending
15062025-09-03T04:16:34.651ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 67eae4df-3eac-4a14-bf2f-9f68e657e627 roles = RoleSet { roles: {} } uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
15072025-09-03T04:16:34.651ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 67eae4df-3eac-4a14-bf2f-9f68e657e627 resource = Database result = Ok(()) uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
15082025-09-03T04:16:34.653ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 2411 local_addr = [fd00:1122:3344:101::b]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:61432 req_id = 67eae4df-3eac-4a14-bf2f-9f68e657e627 response_code = 200 uri = /metrics/collectors/5ae9222c-aa77-4bb7-9fcb-c8d01163fee8/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmYTk5Mjg3Mi1hNjlkLTQ3NjAtYjM3Zi0xYmQ0ZmU0MTI2ZWYifX0%3D
15092025-09-03T04:16:37.267ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
15102025-09-03T04:16:37.267ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
15112025-09-03T04:16:37.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
15122025-09-03T04:16:37.268ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
15132025-09-03T04:16:37.268ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
15142025-09-03T04:16:37.268ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
15152025-09-03T04:16:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
15162025-09-03T04:16:37.270ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1808, tv_nsec: 941476680 } })
15172025-09-03T04:16:37.270ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
15182025-09-03T04:16:37.270ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1808, tv_nsec: 942046654 } })
15192025-09-03T04:16:37.270ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 151 reason = Timeout
15202025-09-03T04:16:37.270ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
15212025-09-03T04:16:37.270ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
15222025-09-03T04:16:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
15232025-09-03T04:16:37.271ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.726137ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
15242025-09-03T04:16:37.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15252025-09-03T04:16:37.275ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
15262025-09-03T04:16:37.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
15272025-09-03T04:16:37.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
15282025-09-03T04:16:37.277ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
15292025-09-03T04:16:37.277ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.504427ms iteration = 151 status = {"chicken_switches_updated":false}
15302025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Timeout
15312025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Timeout
15322025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15332025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15342025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
15352025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
15362025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
15372025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
15382025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15392025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
15402025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 11.75\u{b5}s iteration = 26 status = {"disabled":true,"errors":[],"sps":[]}
15412025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
15422025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
15432025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
15442025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
15452025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
15462025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15472025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
15482025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
15492025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
15502025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
15512025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
15522025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
15532025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
15542025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
15552025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
15562025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
15572025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
15582025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
15592025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
15602025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
15612025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
15622025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15632025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
15642025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Timeout
15652025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
15662025-09-03T04:16:37.293ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
15672025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
15682025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
15692025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
15702025-09-03T04:16:37.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
15712025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15722025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
15732025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
15742025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
15752025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
15762025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
15772025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
15782025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
15792025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
15802025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
15812025-09-03T04:16:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15822025-09-03T04:16:37.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15832025-09-03T04:16:37.295ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15842025-09-03T04:16:37.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15852025-09-03T04:16:37.295ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.011125ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
15862025-09-03T04:16:37.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15872025-09-03T04:16:37.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15882025-09-03T04:16:37.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15892025-09-03T04:16:37.298ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
15902025-09-03T04:16:37.298ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 4.840815ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
15912025-09-03T04:16:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
15922025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15932025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15942025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
15952025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15962025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
15972025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
15982025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
15992025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16002025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 7.124202ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16012025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16022025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
16032025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16042025-09-03T04:16:37.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16052025-09-03T04:16:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
16062025-09-03T04:16:37.301ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16072025-09-03T04:16:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16082025-09-03T04:16:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
16092025-09-03T04:16:37.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16102025-09-03T04:16:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
16112025-09-03T04:16:37.302ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16122025-09-03T04:16:37.303ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1808, tv_nsec: 974488239 } })
16132025-09-03T04:16:37.303ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16142025-09-03T04:16:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16152025-09-03T04:16:37.303ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
16162025-09-03T04:16:37.303ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16172025-09-03T04:16:37.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16182025-09-03T04:16:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16192025-09-03T04:16:37.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
16202025-09-03T04:16:37.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16212025-09-03T04:16:37.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16222025-09-03T04:16:37.305ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
16232025-09-03T04:16:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
16242025-09-03T04:16:37.306ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
16252025-09-03T04:16:37.306ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16262025-09-03T04:16:37.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16272025-09-03T04:16:37.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
16282025-09-03T04:16:37.308ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
16292025-09-03T04:16:37.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
16302025-09-03T04:16:37.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
16312025-09-03T04:16:37.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
16322025-09-03T04:16:37.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
16332025-09-03T04:16:37.320ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
16342025-09-03T04:16:37.320ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
16352025-09-03T04:16:37.320ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16362025-09-03T04:16:37.320ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
16372025-09-03T04:16:37.320ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1940a14-8589-47ec-b010-66fbbc6501ca", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
16382025-09-03T04:16:37.320ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "688579da-85fc-4f2e-8347-9f2fda457db0", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
16392025-09-03T04:16:37.320ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16402025-09-03T04:16:37.320ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
16412025-09-03T04:16:37.323ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16422025-09-03T04:16:37.323ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
16432025-09-03T04:16:37.335ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
16442025-09-03T04:16:37.335ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 42.463157ms iteration = 76 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
16452025-09-03T04:16:37.342ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16462025-09-03T04:16:37.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16472025-09-03T04:16:37.348ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
16482025-09-03T04:16:37.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
16492025-09-03T04:16:37.351ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16502025-09-03T04:16:37.353ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
16512025-09-03T04:16:37.353ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
16522025-09-03T04:16:37.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
16532025-09-03T04:16:37.355ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
16542025-09-03T04:16:37.356ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
16552025-09-03T04:16:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
16562025-09-03T04:16:37.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
16572025-09-03T04:16:37.358ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 64.711799ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
16582025-09-03T04:16:37.359ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 65.404304ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
16592025-09-03T04:16:37.361ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 68.101733ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
16602025-09-03T04:16:37.363ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16612025-09-03T04:16:37.363ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
16622025-09-03T04:16:37.363ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16632025-09-03T04:16:37.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
16642025-09-03T04:16:37.367ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16652025-09-03T04:16:37.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16662025-09-03T04:16:37.370ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16672025-09-03T04:16:37.373ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
16682025-09-03T04:16:37.373ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
16692025-09-03T04:16:37.374ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
16702025-09-03T04:16:37.375ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16712025-09-03T04:16:37.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16722025-09-03T04:16:37.383ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
16732025-09-03T04:16:37.383ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 89.850351ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
16742025-09-03T04:16:37.389ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16752025-09-03T04:16:37.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16762025-09-03T04:16:37.392ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16772025-09-03T04:16:37.393ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 99.872413ms iteration = 28 status = {"physical_disks_added":0}
16782025-09-03T04:16:37.397ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
16792025-09-03T04:16:37.397ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
16802025-09-03T04:16:37.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
16812025-09-03T04:16:37.399ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 106.487951ms iteration = 26 status = 0
16822025-09-03T04:16:37.407ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 113.936035ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
16832025-09-03T04:16:37.424ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16842025-09-03T04:16:37.424ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
16852025-09-03T04:16:37.424ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
16862025-09-03T04:16:37.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
16872025-09-03T04:16:37.428ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
16882025-09-03T04:16:37.428ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
16892025-09-03T04:16:37.428ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 134.509865ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
16902025-09-03T04:16:37.434ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
16912025-09-03T04:16:37.434ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
16922025-09-03T04:16:37.434ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "01592ff3-4a39-443b-86eb-7ed07294aa0d", "content-length": "372", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
16932025-09-03T04:16:37.434ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
16942025-09-03T04:16:37.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
16952025-09-03T04:16:37.444ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
16962025-09-03T04:16:37.444ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 150.56497ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
16972025-09-03T04:16:37.450ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
16982025-09-03T04:16:37.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
16992025-09-03T04:16:37.452ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 185.118151ms iteration = 26 status = null
17002025-09-03T04:16:37.482ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
17012025-09-03T04:16:37.482ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
17022025-09-03T04:16:37.482ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "37736481-5dfe-47fa-bdd8-6ff31fef6cd9", "content-length": "26", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17032025-09-03T04:16:37.482ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
17042025-09-03T04:16:37.484ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
17052025-09-03T04:16:37.485ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
17062025-09-03T04:16:37.485ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
17072025-09-03T04:16:37.485ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9506e718-cd39-4f05-b3d2-8a0789c7bf8a", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17082025-09-03T04:16:37.485ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17092025-09-03T04:16:37.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17102025-09-03T04:16:37.488ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17112025-09-03T04:16:37.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17122025-09-03T04:16:37.488ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 220.326856ms iteration = 26 status = {}
17132025-09-03T04:16:37.491ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
17142025-09-03T04:16:37.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
17152025-09-03T04:16:37.494ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17162025-09-03T04:16:37.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
17172025-09-03T04:16:37.495ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
17182025-09-03T04:16:37.495ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
17192025-09-03T04:16:37.495ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0e7bfded-d927-48ac-9bc2-c210548221a2", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17202025-09-03T04:16:37.496ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17212025-09-03T04:16:37.498ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17222025-09-03T04:16:37.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17232025-09-03T04:16:37.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
17242025-09-03T04:16:37.498ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 204.924944ms iteration = 26 status = {}
17252025-09-03T04:16:37.499ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17262025-09-03T04:16:37.502ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
17272025-09-03T04:16:37.502ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
17282025-09-03T04:16:37.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
17292025-09-03T04:16:37.548ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
17302025-09-03T04:16:37.549ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5bfc651-bb7f-4f80-b79c-e088ceb71ad2", "content-length": "691", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17312025-09-03T04:16:37.549ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
17322025-09-03T04:16:37.549ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
17332025-09-03T04:16:37.549ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
17342025-09-03T04:16:37.550ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 256.721699ms iteration = 26 status = {}
17352025-09-03T04:16:37.586ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17362025-09-03T04:16:37.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17372025-09-03T04:16:37.589ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
17382025-09-03T04:16:37.589ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17392025-09-03T04:16:37.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17402025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
17412025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
17422025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
17432025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
17442025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17452025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17462025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17472025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17482025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
17492025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
17502025-09-03T04:16:37.599ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
17512025-09-03T04:16:37.599ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17522025-09-03T04:16:37.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17532025-09-03T04:16:37.603ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
17542025-09-03T04:16:37.603ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
17552025-09-03T04:16:37.712ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "7ada0767-8cb0-4652-992f-9fa3a92a2c35", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17562025-09-03T04:16:37.712ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
17572025-09-03T04:16:37.712ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
17582025-09-03T04:16:37.718ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a77884-c3b8-4e4c-9df7-739b2ca48fed", "content-length": "682", "date": "Wed, 03 Sep 2025 04:16:37 GMT"} })
17592025-09-03T04:16:37.718ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17602025-09-03T04:16:37.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17612025-09-03T04:16:37.721ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17622025-09-03T04:16:37.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17632025-09-03T04:16:37.724ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17642025-09-03T04:16:37.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17652025-09-03T04:16:37.727ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
17662025-09-03T04:16:37.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
17672025-09-03T04:16:37.734ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 440.427949ms iteration = 26 status = {}
17682025-09-03T04:16:39.654ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 454 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = c0febaa6-553a-4dd3-9aad-e71cc039e093 response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
17692025-09-03T04:16:42.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 152 reason = Timeout
17702025-09-03T04:16:42.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17712025-09-03T04:16:42.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17722025-09-03T04:16:42.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17732025-09-03T04:16:42.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
17742025-09-03T04:16:42.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17752025-09-03T04:16:42.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17762025-09-03T04:16:42.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
17772025-09-03T04:16:42.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 6.665279ms iteration = 152 status = {"chicken_switches_updated":false}
17782025-09-03T04:16:47.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 153 reason = Timeout
17792025-09-03T04:16:47.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17802025-09-03T04:16:47.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17812025-09-03T04:16:47.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17822025-09-03T04:16:47.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
17832025-09-03T04:16:47.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
17842025-09-03T04:16:47.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
17852025-09-03T04:16:47.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
17862025-09-03T04:16:47.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.331483ms iteration = 153 status = {"chicken_switches_updated":false}
17872025-09-03T04:16:47.293ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
17882025-09-03T04:16:47.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17892025-09-03T04:16:47.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17902025-09-03T04:16:47.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
17912025-09-03T04:16:47.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
17922025-09-03T04:16:47.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
17932025-09-03T04:16:47.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17942025-09-03T04:16:47.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
17952025-09-03T04:16:47.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17962025-09-03T04:16:47.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17972025-09-03T04:16:47.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
17982025-09-03T04:16:47.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
17992025-09-03T04:16:47.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18002025-09-03T04:16:47.308ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
18012025-09-03T04:16:47.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18022025-09-03T04:16:47.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18032025-09-03T04:16:47.322ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
18042025-09-03T04:16:47.322ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.97571ms iteration = 77 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
18052025-09-03T04:16:47.337ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18062025-09-03T04:16:47.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18072025-09-03T04:16:47.339ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.396869ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18082025-09-03T04:16:49.655ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 278 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 97c4b1d1-00fd-405e-9ac8-e96da900000a response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
18092025-09-03T04:16:52.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 154 reason = Timeout
18102025-09-03T04:16:52.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18112025-09-03T04:16:52.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18122025-09-03T04:16:52.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18132025-09-03T04:16:52.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
18142025-09-03T04:16:52.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18152025-09-03T04:16:52.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18162025-09-03T04:16:52.279ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
18172025-09-03T04:16:52.279ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.663446ms iteration = 154 status = {"chicken_switches_updated":false}
18182025-09-03T04:16:57.272ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 155 reason = Timeout
18192025-09-03T04:16:57.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18202025-09-03T04:16:57.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18212025-09-03T04:16:57.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18222025-09-03T04:16:57.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
18232025-09-03T04:16:57.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18242025-09-03T04:16:57.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18252025-09-03T04:16:57.280ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
18262025-09-03T04:16:57.280ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.760846ms iteration = 155 status = {"chicken_switches_updated":false}
18272025-09-03T04:16:57.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
18282025-09-03T04:16:57.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18292025-09-03T04:16:57.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18302025-09-03T04:16:57.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
18312025-09-03T04:16:57.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18322025-09-03T04:16:57.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18332025-09-03T04:16:57.296ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
18342025-09-03T04:16:57.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18352025-09-03T04:16:57.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
18362025-09-03T04:16:57.298ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.501425ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
18372025-09-03T04:16:57.300ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
18382025-09-03T04:16:57.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18392025-09-03T04:16:57.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18402025-09-03T04:16:57.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18412025-09-03T04:16:57.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18422025-09-03T04:16:57.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18432025-09-03T04:16:57.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
18442025-09-03T04:16:57.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18452025-09-03T04:16:57.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18462025-09-03T04:16:57.319ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
18472025-09-03T04:16:57.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.298103ms iteration = 78 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
18482025-09-03T04:16:59.655ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 505 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 4cc66ad9-379c-4e41-95dd-6be161020b66 response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
18492025-09-03T04:17:02.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 156 reason = Timeout
18502025-09-03T04:17:02.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18512025-09-03T04:17:02.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18522025-09-03T04:17:02.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18532025-09-03T04:17:02.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
18542025-09-03T04:17:02.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
18552025-09-03T04:17:02.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
18562025-09-03T04:17:02.279ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
18572025-09-03T04:17:02.279ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.425154ms iteration = 156 status = {"chicken_switches_updated":false}
18582025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
18592025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
18602025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
18612025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
18622025-09-03T04:17:07.268ZWARNnexus (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor file = nexus/src/app/background/tasks/blueprint_execution.rs:133 reason = blueprint disabled target_id = 60e73e54-9659-486a-96ba-b69df28d8796
18632025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
18642025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 22.53\u{b5}s iteration = 15 status = {"enabled":false,"target_id":"60e73e54-9659-486a-96ba-b69df28d8796"}
18652025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
18662025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
18672025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
18682025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
18692025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
18702025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18712025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
18722025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
18732025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
18742025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18752025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
18762025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18772025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 27 reason = Timeout
18782025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
18792025-09-03T04:17:07.268ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-09-03 04:07:07.268764415 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
18802025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
18812025-09-03T04:17:07.268ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
18822025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
18832025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18842025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
18852025-09-03T04:17:07.269ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("2085efc3-987d-4d25-bf4e-dbfa977a0d96.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("a853d036-9d99-4741-85fc-3823419b4ac1.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 1838, tv_nsec: 939731320 } })
18862025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
18872025-09-03T04:17:07.269ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0d788cd6-59e1-4eee-8401-73a64a0795db.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("33692814-cb11-4468-bf51-4d6d8418d856.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("808b9b46-4a4d-447d-99d9-09db72c60cdc.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:2::1)) }], valid_until: Instant { tv_sec: 1838, tv_nsec: 939833421 } })
18882025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
18892025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18902025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
18912025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.065028ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
18922025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18932025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } dns_group = internal
18942025-09-03T04:17:07.269ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.212048ms iteration = 14 status = {"addresses":["[fd00:1122:3344:1::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:2::1]:5353"]}
18952025-09-03T04:17:07.270ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18962025-09-03T04:17:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18972025-09-03T04:17:07.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18982025-09-03T04:17:07.271ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1838, tv_nsec: 942006286 } })
18992025-09-03T04:17:07.271ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-09-03 04:07:07.268764415 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
19002025-09-03T04:17:07.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 157 reason = Timeout
19012025-09-03T04:17:07.271ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 2.78938ms iteration = 14 status = {"expiration":"2025-09-03T04:07:07.268764415Z","pruned":[]}
19022025-09-03T04:17:07.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
19032025-09-03T04:17:07.271ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19042025-09-03T04:17:07.271ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
19052025-09-03T04:17:07.271ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1838, tv_nsec: 942446259 } })
19062025-09-03T04:17:07.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
19072025-09-03T04:17:07.272ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19082025-09-03T04:17:07.272ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
19092025-09-03T04:17:07.272ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19102025-09-03T04:17:07.273ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.396622ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
19112025-09-03T04:17:07.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19122025-09-03T04:17:07.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19132025-09-03T04:17:07.275ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19142025-09-03T04:17:07.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
19152025-09-03T04:17:07.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19162025-09-03T04:17:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19172025-09-03T04:17:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
19182025-09-03T04:17:07.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
19192025-09-03T04:17:07.277ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19202025-09-03T04:17:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
19212025-09-03T04:17:07.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
19222025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
19232025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
19242025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
19252025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19262025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19272025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 17 reason = Timeout
19282025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): blueprint planning disabled, doing nothing background_task = blueprint_planner
19292025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 11.7\u{b5}s iteration = 17 status = "Disabled"
19302025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
19312025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
19322025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
19332025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19342025-09-03T04:17:07.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19352025-09-03T04:17:07.279ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
19362025-09-03T04:17:07.279ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.974907ms iteration = 157 status = {"chicken_switches_updated":false}
19372025-09-03T04:17:07.279ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19382025-09-03T04:17:07.279ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19392025-09-03T04:17:07.280ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19402025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
19412025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19422025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
19432025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19442025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19452025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
19462025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19472025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19482025-09-03T04:17:07.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19492025-09-03T04:17:07.286ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19502025-09-03T04:17:07.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
19512025-09-03T04:17:07.287ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
19522025-09-03T04:17:07.287ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
19532025-09-03T04:17:07.287ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19542025-09-03T04:17:07.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19552025-09-03T04:17:07.288ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19562025-09-03T04:17:07.288ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19572025-09-03T04:17:07.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19582025-09-03T04:17:07.290ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19592025-09-03T04:17:07.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
19602025-09-03T04:17:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19612025-09-03T04:17:07.291ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19622025-09-03T04:17:07.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19632025-09-03T04:17:07.293ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = cf400120-81cc-41d9-8da6-7e88a5d7c686 dns_zone_name = control-plane.oxide.internal found_so_far = 59 version = 1
19642025-09-03T04:17:07.293ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
19652025-09-03T04:17:07.293ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-09-03 04:04:24.694355 UTC dns_group = internal generation = 1
19662025-09-03T04:17:07.293ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 24.996861ms iteration = 14 status = {"generation":1}
19672025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
19682025-09-03T04:17:07.294ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:53
19692025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
19702025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
19712025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
19722025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
19732025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
19742025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
19752025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
19762025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
19772025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19782025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
19792025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
19802025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
19812025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19822025-09-03T04:17:07.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
19832025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
19842025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
19852025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
19862025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
19872025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
19882025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
19892025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
19902025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
19912025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
19922025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
19932025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
19942025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
19952025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
19962025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
19972025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
19982025-09-03T04:17:07.295ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
19992025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20002025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
20012025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20022025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
20032025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20042025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
20052025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
20062025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20072025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
20082025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20092025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
20102025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
20112025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
20122025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
20132025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
20142025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
20152025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
20162025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
20172025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 27 reason = Timeout
20182025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 10.61\u{b5}s iteration = 27 status = {"disabled":true,"errors":[],"sps":[]}
20192025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
20202025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
20212025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20222025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
20232025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
20242025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20252025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
20262025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20272025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20282025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
20292025-09-03T04:17:07.295ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20302025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
20312025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
20322025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20332025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
20342025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20352025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
20362025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20372025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
20382025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20392025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
20402025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
20412025-09-03T04:17:07.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
20422025-09-03T04:17:07.297ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20432025-09-03T04:17:07.297ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
20442025-09-03T04:17:07.297ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20452025-09-03T04:17:07.298ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20462025-09-03T04:17:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
20472025-09-03T04:17:07.298ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
20482025-09-03T04:17:07.298ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
20492025-09-03T04:17:07.298ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 2.929981ms iteration = 14 status = {"by_rx":{},"error":null}
20502025-09-03T04:17:07.298ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20512025-09-03T04:17:07.298ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
20522025-09-03T04:17:07.298ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20532025-09-03T04:17:07.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
20542025-09-03T04:17:07.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
20552025-09-03T04:17:07.299ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.924179ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
20562025-09-03T04:17:07.300ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
20572025-09-03T04:17:07.300ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
20582025-09-03T04:17:07.300ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 5.085707ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
20592025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
20602025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
20612025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20622025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20632025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20642025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20652025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
20662025-09-03T04:17:07.301ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
20672025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 6.320496ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
20682025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20692025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20702025-09-03T04:17:07.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
20712025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
20722025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20732025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
20742025-09-03T04:17:07.302ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
20752025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
20762025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
20772025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
20782025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20792025-09-03T04:17:07.302ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20802025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20812025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20822025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
20832025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
20842025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20852025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20862025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
20872025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
20882025-09-03T04:17:07.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20892025-09-03T04:17:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
20902025-09-03T04:17:07.304ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 8.921474ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
20912025-09-03T04:17:07.304ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20922025-09-03T04:17:07.304ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 9.191867ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"186.0.0","globs_reprocessed":{},"no_receivers":[]}
20932025-09-03T04:17:07.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20942025-09-03T04:17:07.304ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20952025-09-03T04:17:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
20962025-09-03T04:17:07.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20972025-09-03T04:17:07.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20982025-09-03T04:17:07.306ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1838, tv_nsec: 976512916 } })
20992025-09-03T04:17:07.306ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
21002025-09-03T04:17:07.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21012025-09-03T04:17:07.307ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21022025-09-03T04:17:07.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21032025-09-03T04:17:07.310ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
21042025-09-03T04:17:07.310ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 14.662666ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
21052025-09-03T04:17:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
21062025-09-03T04:17:07.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21072025-09-03T04:17:07.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
21082025-09-03T04:17:07.323ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21092025-09-03T04:17:07.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21102025-09-03T04:17:07.326ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21112025-09-03T04:17:07.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21122025-09-03T04:17:07.330ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21132025-09-03T04:17:07.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21142025-09-03T04:17:07.333ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21152025-09-03T04:17:07.333ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
21162025-09-03T04:17:07.333ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 0 version = 2
21172025-09-03T04:17:07.333ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21182025-09-03T04:17:07.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21192025-09-03T04:17:07.335ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21202025-09-03T04:17:07.337ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
21212025-09-03T04:17:07.337ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21222025-09-03T04:17:07.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21232025-09-03T04:17:07.340ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21242025-09-03T04:17:07.341ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21252025-09-03T04:17:07.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21262025-09-03T04:17:07.350ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 0a6d548b-6435-4b7d-951e-09ceac6a4eb0 dns_zone_name = oxide.test found_so_far = 4 version = 2
21272025-09-03T04:17:07.350ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21282025-09-03T04:17:07.350ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-09-03 04:04:25.477141 UTC dns_group = external generation = 2
21292025-09-03T04:17:07.350ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 82.099065ms iteration = 14 status = {"generation":2}
21302025-09-03T04:17:07.376ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21312025-09-03T04:17:07.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21322025-09-03T04:17:07.377ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
21332025-09-03T04:17:07.377ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 83.598975ms iteration = 79 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
21342025-09-03T04:17:07.380ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 85.371178ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
21352025-09-03T04:17:07.385ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21362025-09-03T04:17:07.386ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21372025-09-03T04:17:07.388ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21382025-09-03T04:17:07.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21392025-09-03T04:17:07.389ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21402025-09-03T04:17:07.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21412025-09-03T04:17:07.391ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21422025-09-03T04:17:07.394ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 98.802135ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
21432025-09-03T04:17:07.395ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
21442025-09-03T04:17:07.395ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
21452025-09-03T04:17:07.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
21462025-09-03T04:17:07.400ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21472025-09-03T04:17:07.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21482025-09-03T04:17:07.401ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 105.628235ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21492025-09-03T04:17:07.403ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21502025-09-03T04:17:07.404ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21512025-09-03T04:17:07.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21522025-09-03T04:17:07.404ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21532025-09-03T04:17:07.407ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21542025-09-03T04:17:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
21552025-09-03T04:17:07.409ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
21562025-09-03T04:17:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
21572025-09-03T04:17:07.409ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21582025-09-03T04:17:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
21592025-09-03T04:17:07.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21602025-09-03T04:17:07.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
21612025-09-03T04:17:07.412ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
21622025-09-03T04:17:07.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
21632025-09-03T04:17:07.413ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.695007ms iteration = 27 status = 0
21642025-09-03T04:17:07.414ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21652025-09-03T04:17:07.414ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
21662025-09-03T04:17:07.414ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 118.595068ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
21672025-09-03T04:17:07.415ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21682025-09-03T04:17:07.415ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
21692025-09-03T04:17:07.415ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 121.228378ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
21702025-09-03T04:17:07.418ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21712025-09-03T04:17:07.418ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21722025-09-03T04:17:07.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21732025-09-03T04:17:07.429ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-09-03T04:04:05.025624Z, time_modified: 2025-09-03T04:04:05.025624Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, db_silo: Silo { identity: SiloIdentity { id: 1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-09-03T04:04:25.421556Z, time_modified: 2025-09-03T04:04:25.421556Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac" }] }) }
21742025-09-03T04:17:07.429ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 160.314751ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]}},"default_endpoint":{"silo_id":"1c1007bd-6e86-4fcd-a5ed-ccd6a127afb1","tls_certs":["88177654dc1555d3c82c6767dd227356fa644067c4cbd354dce9bb9fa68212ac"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
21752025-09-03T04:17:07.430ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 134.954377ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
21762025-09-03T04:17:07.437ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 141.376714ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
21772025-09-03T04:17:07.438ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 144.244305ms iteration = 29 status = {"physical_disks_added":0}
21782025-09-03T04:17:07.471ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 175.554871ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
21792025-09-03T04:17:07.475ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
21802025-09-03T04:17:07.475ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:138 found = 0 not_ready_to_be_deleted = 0
21812025-09-03T04:17:07.475ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 180.949711ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
21822025-09-03T04:17:07.508ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
21832025-09-03T04:17:07.508ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21842025-09-03T04:17:07.509ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5736d173-1cbe-4f52-b562-79eafc35ad48", "content-length": "26", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
21852025-09-03T04:17:07.509ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21862025-09-03T04:17:07.510ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21872025-09-03T04:17:07.524ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
21882025-09-03T04:17:07.524ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21892025-09-03T04:17:07.524ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "35cc462d-9382-430a-aeef-ae73554e338e", "content-length": "26", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
21902025-09-03T04:17:07.524ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21912025-09-03T04:17:07.526ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
21922025-09-03T04:17:07.590ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
21932025-09-03T04:17:07.627ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
21942025-09-03T04:17:07.663ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
21952025-09-03T04:17:07.663ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
21962025-09-03T04:17:07.663ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a47bd7d4-20f5-4595-9b1a-7652018f7c2a", "content-length": "26", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
21972025-09-03T04:17:07.663ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
21982025-09-03T04:17:07.666ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
21992025-09-03T04:17:07.717ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 448.626778ms iteration = 27 status = {}
22002025-09-03T04:17:07.720ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
22012025-09-03T04:17:07.721ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6be9ac4a-e20f-4cc9-b8c1-80641c37bc50", "content-length": "691", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
22022025-09-03T04:17:07.721ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
22032025-09-03T04:17:07.721ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
22042025-09-03T04:17:07.721ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
22052025-09-03T04:17:07.723ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 427.569656ms iteration = 27 status = {}
22062025-09-03T04:17:07.752ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
22072025-09-03T04:17:07.752ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22082025-09-03T04:17:07.753ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "dad3f0d9-f17b-4df6-9044-18ceda3b8a04", "content-length": "372", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
22092025-09-03T04:17:07.753ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22102025-09-03T04:17:07.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22112025-09-03T04:17:07.766ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
22122025-09-03T04:17:07.766ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22132025-09-03T04:17:07.766ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "36c228a0-4720-4e4e-b7e1-7bb6258cd222", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
22142025-09-03T04:17:07.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22152025-09-03T04:17:07.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22162025-09-03T04:17:07.769ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22172025-09-03T04:17:07.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22182025-09-03T04:17:07.773ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22192025-09-03T04:17:07.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22202025-09-03T04:17:07.776ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22212025-09-03T04:17:07.776ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22222025-09-03T04:17:07.777ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3bbd1465-32ef-42ac-b1f8-67c93b80ba30", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
22232025-09-03T04:17:07.778ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 483.39414ms iteration = 27 status = {}
22242025-09-03T04:17:07.810ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
22252025-09-03T04:17:07.810ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
22262025-09-03T04:17:07.810ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "206c3176-3e7d-44fa-bc41-a3a033d3f370", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22272025-09-03T04:17:07.810ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22282025-09-03T04:17:07.812ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a661bc67-6926-4964-b562-e27c40e22986", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22292025-09-03T04:17:07.812ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22302025-09-03T04:17:07.814ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
22312025-09-03T04:17:07.814ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 545.582421ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
22322025-09-03T04:17:07.817ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
22332025-09-03T04:17:07.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
22342025-09-03T04:17:07.819ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 550.918269ms iteration = 27 status = null
22352025-09-03T04:17:07.871ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
22362025-09-03T04:17:07.872ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "12c2d9d5-a872-4a02-b5ec-670cd750d844", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22372025-09-03T04:17:07.872ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22382025-09-03T04:17:07.873ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "e8711feb-cbb0-4999-ad73-012d7eaae1f3", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22392025-09-03T04:17:07.873ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22402025-09-03T04:17:07.875ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "e0933cdb-4015-40ba-980a-edc824d3c6ed", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22412025-09-03T04:17:07.875ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22422025-09-03T04:17:07.877ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:1::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:2::1]:5353] }
22432025-09-03T04:17:07.877ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 608.849168ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
22442025-09-03T04:17:07.883ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22452025-09-03T04:17:07.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22462025-09-03T04:17:07.885ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
22472025-09-03T04:17:07.885ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22482025-09-03T04:17:07.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22492025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
22502025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
22512025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
22522025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
22532025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22542025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22552025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22562025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
22572025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
22582025-09-03T04:17:07.895ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
22592025-09-03T04:17:07.896ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
22602025-09-03T04:17:07.896ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22612025-09-03T04:17:07.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22622025-09-03T04:17:07.898ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
22632025-09-03T04:17:07.898ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
22642025-09-03T04:17:08.086ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b11ff717-a668-4b62-98f5-1226fa16da68", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
22652025-09-03T04:17:08.086ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
22662025-09-03T04:17:08.086ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
22672025-09-03T04:17:08.092ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f824760b-0276-4a15-904a-d6e092441019", "content-length": "682", "date": "Wed, 03 Sep 2025 04:17:07 GMT"} })
22682025-09-03T04:17:08.092ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22692025-09-03T04:17:08.092ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22702025-09-03T04:17:08.095ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22712025-09-03T04:17:08.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22722025-09-03T04:17:08.099ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22732025-09-03T04:17:08.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22742025-09-03T04:17:08.102ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22752025-09-03T04:17:08.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22762025-09-03T04:17:08.109ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 813.54935ms iteration = 27 status = {}
22772025-09-03T04:17:09.655ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 301 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = c041b9d0-4440-4dbf-b90f-85c80735697d response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
22782025-09-03T04:17:12.272ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 158 reason = Timeout
22792025-09-03T04:17:12.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
22802025-09-03T04:17:12.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
22812025-09-03T04:17:12.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22822025-09-03T04:17:12.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
22832025-09-03T04:17:12.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
22842025-09-03T04:17:12.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
22852025-09-03T04:17:12.280ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
22862025-09-03T04:17:12.280ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.29682ms iteration = 158 status = {"chicken_switches_updated":false}
22872025-09-03T04:17:17.272ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 159 reason = Timeout
22882025-09-03T04:17:17.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
22892025-09-03T04:17:17.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
22902025-09-03T04:17:17.274ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22912025-09-03T04:17:17.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
22922025-09-03T04:17:17.277ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
22932025-09-03T04:17:17.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
22942025-09-03T04:17:17.280ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
22952025-09-03T04:17:17.280ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.818806ms iteration = 159 status = {"chicken_switches_updated":false}
22962025-09-03T04:17:17.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
22972025-09-03T04:17:17.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
22982025-09-03T04:17:17.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
22992025-09-03T04:17:17.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
23002025-09-03T04:17:17.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23012025-09-03T04:17:17.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23022025-09-03T04:17:17.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23032025-09-03T04:17:17.297ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23042025-09-03T04:17:17.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23052025-09-03T04:17:17.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23062025-09-03T04:17:17.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23072025-09-03T04:17:17.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23082025-09-03T04:17:17.299ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.668144ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23092025-09-03T04:17:17.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23102025-09-03T04:17:17.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23112025-09-03T04:17:17.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23122025-09-03T04:17:17.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
23132025-09-03T04:17:17.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23142025-09-03T04:17:17.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23152025-09-03T04:17:17.319ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
23162025-09-03T04:17:17.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.704099ms iteration = 80 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
23172025-09-03T04:17:19.655ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 300 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = 3de5cfff-dd9b-4df2-ae96-1c8371bd8075 response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
23182025-09-03T04:17:22.272ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 160 reason = Timeout
23192025-09-03T04:17:22.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23202025-09-03T04:17:22.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23212025-09-03T04:17:22.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23222025-09-03T04:17:22.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
23232025-09-03T04:17:22.278ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23242025-09-03T04:17:22.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23252025-09-03T04:17:22.281ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
23262025-09-03T04:17:22.281ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 8.602603ms iteration = 160 status = {"chicken_switches_updated":false}
23272025-09-03T04:17:27.272ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 161 reason = Timeout
23282025-09-03T04:17:27.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23292025-09-03T04:17:27.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23302025-09-03T04:17:27.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23312025-09-03T04:17:27.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
23322025-09-03T04:17:27.278ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23332025-09-03T04:17:27.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23342025-09-03T04:17:27.280ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
23352025-09-03T04:17:27.280ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.735656ms iteration = 161 status = {"chicken_switches_updated":false}
23362025-09-03T04:17:27.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
23372025-09-03T04:17:27.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23382025-09-03T04:17:27.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23392025-09-03T04:17:27.295ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
23402025-09-03T04:17:27.295ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23412025-09-03T04:17:27.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23422025-09-03T04:17:27.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23432025-09-03T04:17:27.297ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23442025-09-03T04:17:27.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23452025-09-03T04:17:27.299ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.312842ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23462025-09-03T04:17:27.299ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23472025-09-03T04:17:27.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23482025-09-03T04:17:27.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23492025-09-03T04:17:27.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23502025-09-03T04:17:27.300ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23512025-09-03T04:17:27.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23522025-09-03T04:17:27.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
23532025-09-03T04:17:27.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23542025-09-03T04:17:27.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23552025-09-03T04:17:27.319ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
23562025-09-03T04:17:27.319ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.986301ms iteration = 81 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
23572025-09-03T04:17:29.655ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.3/src/server.rs:867 latency_us = 454 local_addr = [fd00:1122:3344:101::b]:43648 method = GET remote_addr = [fd00:1122:3344:101::d]:62803 req_id = b5ff07e2-4f6b-4f1e-8291-da9d807c3e4e response_code = 200 uri = /fa992872-a69d-4760-b37f-1bd4fe4126ef
23582025-09-03T04:17:32.272ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 162 reason = Timeout
23592025-09-03T04:17:32.272ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23602025-09-03T04:17:32.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23612025-09-03T04:17:32.275ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23622025-09-03T04:17:32.278ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
23632025-09-03T04:17:32.279ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23642025-09-03T04:17:32.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23652025-09-03T04:17:32.281ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
23662025-09-03T04:17:32.281ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 9.071695ms iteration = 162 status = {"chicken_switches_updated":false}
23672025-09-03T04:17:37.268ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 28 reason = Timeout
23682025-09-03T04:17:37.268ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
23692025-09-03T04:17:37.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
23702025-09-03T04:17:37.268ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
23712025-09-03T04:17:37.268ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
23722025-09-03T04:17:37.268ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
23732025-09-03T04:17:37.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
23742025-09-03T04:17:37.271ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1868, tv_nsec: 942031281 } })
23752025-09-03T04:17:37.271ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1868, tv_nsec: 942033391 } })
23762025-09-03T04:17:37.271ZDEBGnexus (BackgroundTasks): activating background_task = chicken_switches_watcher iteration = 163 reason = Timeout
23772025-09-03T04:17:37.271ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23782025-09-03T04:17:37.271ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1206
23792025-09-03T04:17:37.271ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1206
23802025-09-03T04:17:37.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23812025-09-03T04:17:37.272ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 3.593186ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
23822025-09-03T04:17:37.273ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23832025-09-03T04:17:37.276ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = BlueprintConfig result = Ok(())
23842025-09-03T04:17:37.276ZDEBGnexus (BackgroundTasks): roles background_task = chicken_switches_watcher roles = RoleSet { roles: {} }
23852025-09-03T04:17:37.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = chicken_switches_watcher resource = Database result = Ok(())
23862025-09-03T04:17:37.278ZDEBGnexus (BackgroundTasks): chicken switches load complete background_task = chicken_switches_watcher
23872025-09-03T04:17:37.278ZDEBGnexus (BackgroundTasks): activation complete background_task = chicken_switches_watcher elapsed = 7.489944ms iteration = 163 status = {"chicken_switches_updated":false}
23882025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
23892025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
23902025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
23912025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
23922025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
23932025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
23942025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
23952025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 28 reason = Timeout
23962025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 9.32\u{b5}s iteration = 28 status = {"disabled":true,"errors":[],"sps":[]}
23972025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
23982025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
23992025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24002025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
24012025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
24022025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
24032025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24042025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
24052025-09-03T04:17:37.294ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
24062025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
24072025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
24082025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24092025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
24102025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
24112025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
24122025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
24132025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24142025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
24152025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
24162025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
24172025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24182025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
24192025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
24202025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
24212025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24222025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
24232025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24242025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24252025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24262025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
24272025-09-03T04:17:37.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24282025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24292025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
24302025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
24312025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
24322025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
24332025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
24342025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
24352025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24362025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24372025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
24382025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24392025-09-03T04:17:37.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
24402025-09-03T04:17:37.296ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24412025-09-03T04:17:37.296ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24422025-09-03T04:17:37.296ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 2.443008ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
24432025-09-03T04:17:37.296ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24442025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.568078ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
24452025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24462025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24472025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24482025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24492025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24502025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24512025-09-03T04:17:37.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24522025-09-03T04:17:37.298ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
24532025-09-03T04:17:37.298ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
24542025-09-03T04:17:37.298ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
24552025-09-03T04:17:37.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
24562025-09-03T04:17:37.298ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 4.22453ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24572025-09-03T04:17:37.298ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
24582025-09-03T04:17:37.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
24592025-09-03T04:17:37.299ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
24602025-09-03T04:17:37.299ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 4.991217ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
24612025-09-03T04:17:37.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
24622025-09-03T04:17:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
24632025-09-03T04:17:37.301ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
24642025-09-03T04:17:37.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
24652025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 7.719965ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
24662025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
24672025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
24682025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24692025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24702025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
24712025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24722025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24732025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24742025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
24752025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
24762025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24772025-09-03T04:17:37.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
24782025-09-03T04:17:37.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24792025-09-03T04:17:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
24802025-09-03T04:17:37.306ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24812025-09-03T04:17:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24822025-09-03T04:17:37.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24832025-09-03T04:17:37.307ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24842025-09-03T04:17:37.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24852025-09-03T04:17:37.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24862025-09-03T04:17:37.308ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24872025-09-03T04:17:37.308ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-6ba5229c-c111-45f6-88aa-7f00b514c681.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 1868, tv_nsec: 979238611 } })
24882025-09-03T04:17:37.308ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
24892025-09-03T04:17:37.308ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1206 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
24902025-09-03T04:17:37.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
24912025-09-03T04:17:37.311ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
24922025-09-03T04:17:37.311ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
24932025-09-03T04:17:37.311ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:550 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
24942025-09-03T04:17:37.311ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 17.009323ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
24952025-09-03T04:17:37.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
24962025-09-03T04:17:37.312ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 60e73e54-9659-486a-96ba-b69df28d8796, lookup_type: ById(60e73e54-9659-486a-96ba-b69df28d8796) } result = Ok(())
24972025-09-03T04:17:37.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24982025-09-03T04:17:37.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24992025-09-03T04:17:37.321ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
25002025-09-03T04:17:37.321ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25012025-09-03T04:17:37.321ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "68eee869-2734-4850-a6eb-5b8b923c5261", "content-length": "26", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
25022025-09-03T04:17:37.321ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = bfd_manager file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25032025-09-03T04:17:37.323ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25042025-09-03T04:17:37.328ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1214 zone_address = fd00:1122:3344:101::2
25052025-09-03T04:17:37.328ZDEBGnexus (MgsClient): client request background_task = nat_garbage_collector body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25062025-09-03T04:17:37.329ZDEBGnexus (MgsClient): client response background_task = nat_garbage_collector result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "375e1bd7-5ec0-4a21-bfc1-e45b0fb1020d", "content-length": "26", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
25072025-09-03T04:17:37.329ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1217 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25082025-09-03T04:17:37.331ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = nat_garbage_collector file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
25092025-09-03T04:17:37.344ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 60e73e54-9659-486a-96ba-b69df28d8796 original_time_created = 2025-09-03 04:03:56.692551 UTC target_id = 60e73e54-9659-486a-96ba-b69df28d8796 time_created = 2025-09-03 04:03:56.692551 UTC
25102025-09-03T04:17:37.344ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 50.408935ms iteration = 82 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"60e73e54-9659-486a-96ba-b69df28d8796","time_created":"2025-09-03T04:03:56.692551Z"}
25112025-09-03T04:17:37.360ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25122025-09-03T04:17:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25132025-09-03T04:17:37.362ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
25142025-09-03T04:17:37.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
25152025-09-03T04:17:37.363ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25162025-09-03T04:17:37.363ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
25172025-09-03T04:17:37.363ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25182025-09-03T04:17:37.364ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25192025-09-03T04:17:37.370ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 75.707079ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
25202025-09-03T04:17:37.371ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
25212025-09-03T04:17:37.371ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25222025-09-03T04:17:37.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
25232025-09-03T04:17:37.371ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25242025-09-03T04:17:37.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25252025-09-03T04:17:37.374ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:407
25262025-09-03T04:17:37.374ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 79.808868ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
25272025-09-03T04:17:37.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
25282025-09-03T04:17:37.376ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25292025-09-03T04:17:37.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25302025-09-03T04:17:37.378ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25312025-09-03T04:17:37.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25322025-09-03T04:17:37.379ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25332025-09-03T04:17:37.382ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25342025-09-03T04:17:37.382ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
25352025-09-03T04:17:37.382ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25362025-09-03T04:17:37.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25372025-09-03T04:17:37.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25382025-09-03T04:17:37.389ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 94.948997ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
25392025-09-03T04:17:37.394ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25402025-09-03T04:17:37.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 6ba5229c-c111-45f6-88aa-7f00b514c681, lookup_type: ById(6ba5229c-c111-45f6-88aa-7f00b514c681) } result = Ok(())
25412025-09-03T04:17:37.399ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25422025-09-03T04:17:37.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
25432025-09-03T04:17:37.402ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 108.311274ms iteration = 28 status = 0
25442025-09-03T04:17:37.423ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 128.613151ms iteration = 30 status = {"physical_disks_added":0}
25452025-09-03T04:17:37.434ZINFOnexus (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1214 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 zone_address = fd00:1122:3344:101::2
25462025-09-03T04:17:37.434ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
25472025-09-03T04:17:37.436ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a32b775-4d83-427d-90bc-6918cd530273", "content-length": "26", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
25482025-09-03T04:17:37.436ZINFOnexus (BackgroundTasks): identified switch slot for dendrite zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1217 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
25492025-09-03T04:17:37.437ZINFOnexus (BackgroundTasks): completed mapping dendrite zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1251 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
25502025-09-03T04:17:37.449ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
25512025-09-03T04:17:37.449ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
25522025-09-03T04:17:37.449ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 154.495509ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
25532025-09-03T04:17:37.474ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
25542025-09-03T04:17:37.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
25552025-09-03T04:17:37.476ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 208.313308ms iteration = 28 status = null
25562025-09-03T04:17:37.490ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25572025-09-03T04:17:37.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
25582025-09-03T04:17:37.498ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25592025-09-03T04:17:37.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
25602025-09-03T04:17:37.500ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25612025-09-03T04:17:37.503ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25622025-09-03T04:17:37.503ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25632025-09-03T04:17:37.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
25642025-09-03T04:17:37.505ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25652025-09-03T04:17:37.507ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
25662025-09-03T04:17:37.507ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
25672025-09-03T04:17:37.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
25682025-09-03T04:17:37.533ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 264.974499ms iteration = 28 status = {}
25692025-09-03T04:17:37.555ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
25702025-09-03T04:17:37.555ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25712025-09-03T04:17:37.555ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ab07f86-d10f-4789-b09b-db058bdfd3c3", "content-length": "372", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
25722025-09-03T04:17:37.555ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25732025-09-03T04:17:37.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25742025-09-03T04:17:37.568ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = buskin
25752025-09-03T04:17:37.568ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
25762025-09-03T04:17:37.569ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5f21d1e4-d0f3-4bbd-8287-4f3f94512373", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
25772025-09-03T04:17:37.569ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25782025-09-03T04:17:37.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25792025-09-03T04:17:37.571ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25802025-09-03T04:17:37.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25812025-09-03T04:17:37.575ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
25822025-09-03T04:17:37.575ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
25832025-09-03T04:17:37.577ZINFOnexus (BackgroundTasks): VPC route manager sled 6ba5229c-c111-45f6-88aa-7f00b514c681 push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
25842025-09-03T04:17:37.577ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
25852025-09-03T04:17:37.577ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ccedc1d1-2d1b-42c9-bb4a-ebf8f26be2e6", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
25862025-09-03T04:17:37.579ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 284.920253ms iteration = 28 status = {}
25872025-09-03T04:17:37.583ZDEBGnexus (BackgroundTasks): client request SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
25882025-09-03T04:17:37.584ZDEBGnexus (BackgroundTasks): client response SledAgent = 6ba5229c-c111-45f6-88aa-7f00b514c681 background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebe91220-147c-4889-bd8f-cd7b4e496315", "content-length": "691", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
25892025-09-03T04:17:37.584ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:102 interfaces = {} sled = buskin
25902025-09-03T04:17:37.584ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:114 mappings = [] sled = buskin
25912025-09-03T04:17:37.584ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:127 mappings = [] sled = buskin
25922025-09-03T04:17:37.586ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 291.625701ms iteration = 28 status = {}
25932025-09-03T04:17:37.623ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25942025-09-03T04:17:37.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25952025-09-03T04:17:37.626ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 6d67bb72-7d81-4e35-9440-7d645a356abd, rack_id: b1c08780-6eb9-4097-acea-f38d0539ff40, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(2c3aab11-bea9-412a-a4d5-9f98fa9ac796) } rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
25962025-09-03T04:17:37.626ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
25972025-09-03T04:17:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
25982025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1832 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.1.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
25992025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1859 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 2c3aab11-bea9-412a-a4d5-9f98fa9ac796
26002025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:378 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
26012025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:382 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: {(10.151.1.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
26022025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:391 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26032025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:397 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26042025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:402 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26052025-09-03T04:17:37.639ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2006 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26062025-09-03T04:17:37.640ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:408 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
26072025-09-03T04:17:37.640ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2042 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.1.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
26082025-09-03T04:17:37.640ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:417 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40
26092025-09-03T04:17:37.640ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26102025-09-03T04:17:37.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26112025-09-03T04:17:37.642ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.1.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:466 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch_location = Switch0
26122025-09-03T04:17:37.642ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
26132025-09-03T04:17:37.805ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "243b926f-63bf-4fe8-a055-5df217c83b08", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
26142025-09-03T04:17:37.805ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:769 rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 switch = Switch0
26152025-09-03T04:17:37.805ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
26162025-09-03T04:17:37.810ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = b1c08780-6eb9-4097-acea-f38d0539ff40 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "554be359-5970-4d56-ada7-37fe5763e26a", "content-length": "682", "date": "Wed, 03 Sep 2025 04:17:37 GMT"} })
26172025-09-03T04:17:37.810ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26182025-09-03T04:17:37.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26192025-09-03T04:17:37.814ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26202025-09-03T04:17:37.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26212025-09-03T04:17:37.818ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26222025-09-03T04:17:37.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26232025-09-03T04:17:37.820ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26242025-09-03T04:17:37.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26252025-09-03T04:17:37.827ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 532.834798ms iteration = 28 status = {}