Max level shown:
12025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Timeout
22025-11-13T08:00:02.607ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
32025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 26.851\u{b5}s iteration = 3 status = {"error":"no config"}
42025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Timeout
52025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Timeout
62025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
72025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
82025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Timeout
92025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
102025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
112025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Timeout
122025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 3 reason = Timeout
132025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
142025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 2 reason = Timeout
152025-11-13T08:00:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:50:02.607208336 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
162025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
172025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Timeout
182025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Timeout
192025-11-13T08:00:02.607ZWARNnexus (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:69 reason = no config
202025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 10\u{b5}s iteration = 3 status = {"error":"no config"}
212025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 3 reason = Timeout
222025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
232025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 3 reason = Timeout
242025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
252025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
262025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
272025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
282025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 3 reason = Timeout
292025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
302025-11-13T08:00:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
312025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
322025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
332025-11-13T08:00:02.607ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = ExternalNetworking step id = Ensure
342025-11-13T08:00:02.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
352025-11-13T08:00:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
362025-11-13T08:00:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6010, tv_nsec: 960539459 } })
372025-11-13T08:00:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6010, tv_nsec: 960551280 } })
382025-11-13T08:00:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
392025-11-13T08:00:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
402025-11-13T08:00:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.679603ms iteration = 2 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
412025-11-13T08:00:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
422025-11-13T08:00:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.680033ms iteration = 2 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
432025-11-13T08:00:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
442025-11-13T08:00:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
452025-11-13T08:00:02.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
462025-11-13T08:00:02.614ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6010, tv_nsec: 966597369 } })
472025-11-13T08:00:02.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
482025-11-13T08:00:02.615ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
492025-11-13T08:00:02.615ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
502025-11-13T08:00:02.616ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:50:02.607208336 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
512025-11-13T08:00:02.616ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 8.808773ms iteration = 2 status = {"expiration":"2025-11-13T07:50:02.607208336Z","pruned":[]}
522025-11-13T08:00:02.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
532025-11-13T08:00:02.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
542025-11-13T08:00:02.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
552025-11-13T08:00:02.616ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6010, tv_nsec: 969275721 } })
562025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 7 reason = Timeout
572025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 6 reason = Timeout
582025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
592025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
602025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 7 reason = Timeout
612025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
622025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 13 reason = Timeout
632025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 2 reason = Timeout
642025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
652025-11-13T08:00:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
662025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
672025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 4 reason = Timeout
682025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
692025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 7 reason = Timeout
702025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
712025-11-13T08:00:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
722025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
732025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
742025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
752025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
762025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
772025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
782025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
792025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
802025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
812025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.18623ms iteration = 3 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
822025-11-13T08:00:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
832025-11-13T08:00:02.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
842025-11-13T08:00:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
852025-11-13T08:00:02.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
862025-11-13T08:00:02.621ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
872025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
882025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
892025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
902025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
912025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
922025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
932025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
942025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
952025-11-13T08:00:02.622ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
962025-11-13T08:00:02.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
972025-11-13T08:00:02.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
982025-11-13T08:00:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
992025-11-13T08:00:02.627ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
1002025-11-13T08:00:02.627ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1012025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
1022025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
1032025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
1042025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1052025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1062025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
1072025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1082025-11-13T08:00:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1092025-11-13T08:00:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
1102025-11-13T08:00:02.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1112025-11-13T08:00:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1122025-11-13T08:00:02.630ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
1132025-11-13T08:00:02.630ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
1142025-11-13T08:00:02.630ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
1152025-11-13T08:00:02.630ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1162025-11-13T08:00:02.631ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1172025-11-13T08:00:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1182025-11-13T08:00:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1192025-11-13T08:00:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
1202025-11-13T08:00:02.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
1212025-11-13T08:00:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
1222025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
1232025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
1242025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1252025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1262025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1272025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
1282025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1292025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
1302025-11-13T08:00:02.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 16.318921ms iteration = 13 status = {"config_updated":false}
1312025-11-13T08:00:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1322025-11-13T08:00:02.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
1332025-11-13T08:00:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
1342025-11-13T08:00:02.640ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
1352025-11-13T08:00:02.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
1362025-11-13T08:00:02.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
1372025-11-13T08:00:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
1382025-11-13T08:00:02.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
1392025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 2 reason = Timeout
1402025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
1412025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 3 reason = Timeout
1422025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
1432025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 3 reason = Timeout
1442025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1452025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 3 reason = Timeout
1462025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
1472025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
1482025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
1492025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
1502025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 3 reason = Timeout
1512025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
1522025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
1532025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 2 reason = Timeout
1542025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
1552025-11-13T08:00:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
1562025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 3 reason = Timeout
1572025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 7 reason = Timeout
1582025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
1592025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
1602025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 3 reason = Timeout
1612025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
1622025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
1632025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 4 reason = Timeout
1642025-11-13T08:00:02.643ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
1652025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
1662025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
1672025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
1682025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
1692025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
1702025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
1712025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
1722025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
1732025-11-13T08:00:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
1742025-11-13T08:00:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 2 reason = Timeout
1752025-11-13T08:00:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
1762025-11-13T08:00:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
1772025-11-13T08:00:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
1782025-11-13T08:00:02.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
1792025-11-13T08:00:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
1802025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 2 reason = Timeout
1812025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
1822025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 3 reason = Timeout
1832025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
1842025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
1852025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 3 reason = Timeout
1862025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 5 reason = Timeout
1872025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
1882025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 3 reason = Timeout
1892025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
1902025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 5 reason = Timeout
1912025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
1922025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 3 reason = Timeout
1932025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
1942025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 2 reason = Timeout
1952025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 3 reason = Timeout
1962025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
1972025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
1982025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 3 reason = Timeout
1992025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
2002025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2012025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2022025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
2032025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
2042025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
2052025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
2062025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
2072025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
2082025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
2092025-11-13T08:00:02.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6011, tv_nsec: 911853 } })
2102025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
2112025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 2 reason = Timeout
2122025-11-13T08:00:02.648ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
2132025-11-13T08:00:02.648ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
2142025-11-13T08:00:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
2152025-11-13T08:00:02.653ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2162025-11-13T08:00:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
2172025-11-13T08:00:02.657ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
2182025-11-13T08:00:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
2192025-11-13T08:00:02.658ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 13.071907ms iteration = 2 status = {"errors":[],"found_port_ok":[]}
2202025-11-13T08:00:02.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2212025-11-13T08:00:02.674ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
2222025-11-13T08:00:02.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
2232025-11-13T08:00:02.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
2242025-11-13T08:00:02.698ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
2252025-11-13T08:00:02.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
2262025-11-13T08:00:02.702ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2272025-11-13T08:00:02.707ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
2282025-11-13T08:00:02.707ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
2292025-11-13T08:00:02.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
2302025-11-13T08:00:02.738ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
2312025-11-13T08:00:02.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2322025-11-13T08:00:02.740ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 133.504484ms iteration = 3 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
2332025-11-13T08:00:02.741ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2342025-11-13T08:00:02.744ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2352025-11-13T08:00:02.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
2362025-11-13T08:00:02.747ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
2372025-11-13T08:00:02.747ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
2382025-11-13T08:00:02.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
2392025-11-13T08:00:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
2402025-11-13T08:00:02.750ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
2412025-11-13T08:00:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
2422025-11-13T08:00:02.752ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
2432025-11-13T08:00:02.752ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
2442025-11-13T08:00:02.752ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 134.239419ms iteration = 2 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
2452025-11-13T08:00:02.758ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
2462025-11-13T08:00:02.763ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
2472025-11-13T08:00:02.763ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 145.284467ms iteration = 7 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
2482025-11-13T08:00:02.764ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
2492025-11-13T08:00:02.764ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
2502025-11-13T08:00:02.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
2512025-11-13T08:00:02.765ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
2522025-11-13T08:00:02.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
2532025-11-13T08:00:02.766ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
2542025-11-13T08:00:02.766ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
2552025-11-13T08:00:02.766ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
2562025-11-13T08:00:02.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
2572025-11-13T08:00:02.772ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
2582025-11-13T08:00:02.774ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
2592025-11-13T08:00:02.775ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2602025-11-13T08:00:02.777ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
2612025-11-13T08:00:02.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
2622025-11-13T08:00:02.779ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
2632025-11-13T08:00:02.779ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
2642025-11-13T08:00:02.779ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
2652025-11-13T08:00:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
2662025-11-13T08:00:02.780ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
2672025-11-13T08:00:02.780ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 132.526575ms iteration = 2 status = {"by_rx":{},"error":null}
2682025-11-13T08:00:02.780ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
2692025-11-13T08:00:02.780ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
2702025-11-13T08:00:02.781ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "6932343f-2b4e-49ba-978e-e0e99633bb75", "content-length": "78", "date": "Thu, 13 Nov 2025 08:00:02 GMT"} })
2712025-11-13T08:00:02.781ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
2722025-11-13T08:00:02.781ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
2732025-11-13T08:00:02.781ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
2742025-11-13T08:00:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
2752025-11-13T08:00:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
2762025-11-13T08:00:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
2772025-11-13T08:00:02.783ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 135.408452ms iteration = 3 status = {"errors":[],"requests_created_ok":[]}
2782025-11-13T08:00:02.793ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
2792025-11-13T08:00:02.794ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 151.775164ms iteration = 3 status = 0
2802025-11-13T08:00:02.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
2812025-11-13T08:00:02.796ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
2822025-11-13T08:00:02.798ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
2832025-11-13T08:00:02.799ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
2842025-11-13T08:00:02.799ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
2852025-11-13T08:00:02.799ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 157.504317ms iteration = 2 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
2862025-11-13T08:00:02.801ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = badd5a66-f8f5-414c-9527-1e9f4eb27e70 original_time_started = 2025-11-13 07:59:23.683558 UTC
2872025-11-13T08:00:02.801ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 182.59758ms iteration = 7 status = {"Loaded":{"collection_id":"badd5a66-f8f5-414c-9527-1e9f4eb27e70","time_loaded":"2025-11-13T08:00:02.618711092Z","time_started":"2025-11-13T07:59:23.683558Z"}}
2882025-11-13T08:00:02.805ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
2892025-11-13T08:00:02.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
2902025-11-13T08:00:02.812ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2912025-11-13T08:00:02.812ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
2922025-11-13T08:00:02.813ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
2932025-11-13T08:00:02.814ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
2942025-11-13T08:00:02.814ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
2952025-11-13T08:00:02.814ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
2962025-11-13T08:00:02.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
2972025-11-13T08:00:02.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
2982025-11-13T08:00:02.814ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
2992025-11-13T08:00:02.816ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
3002025-11-13T08:00:02.816ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
3012025-11-13T08:00:02.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
3022025-11-13T08:00:02.818ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3032025-11-13T08:00:02.820ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
3042025-11-13T08:00:02.824ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3052025-11-13T08:00:02.825ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
3062025-11-13T08:00:02.825ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
3072025-11-13T08:00:02.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
3082025-11-13T08:00:02.825ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
3092025-11-13T08:00:02.825ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
3102025-11-13T08:00:02.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
3112025-11-13T08:00:02.827ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3122025-11-13T08:00:02.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3132025-11-13T08:00:02.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
3142025-11-13T08:00:02.829ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
3152025-11-13T08:00:02.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
3162025-11-13T08:00:02.831ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
3172025-11-13T08:00:02.831ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
3182025-11-13T08:00:02.832ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "236e43a1-406c-4de2-a767-805da1fe0b8e", "content-length": "26", "date": "Thu, 13 Nov 2025 08:00:02 GMT"} })
3192025-11-13T08:00:02.832ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
3202025-11-13T08:00:02.834ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
3212025-11-13T08:00:02.838ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
3222025-11-13T08:00:02.840ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
3232025-11-13T08:00:02.842ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 194.914252ms iteration = 2 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
3242025-11-13T08:00:02.842ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 200.080363ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
3252025-11-13T08:00:02.843ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
3262025-11-13T08:00:02.844ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
3272025-11-13T08:00:02.845ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = SledList step id = Fetch
3282025-11-13T08:00:02.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3292025-11-13T08:00:02.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3302025-11-13T08:00:02.847ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3312025-11-13T08:00:02.850ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3322025-11-13T08:00:02.853ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
3332025-11-13T08:00:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
3342025-11-13T08:00:02.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
3352025-11-13T08:00:02.854ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
3362025-11-13T08:00:02.855ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
3372025-11-13T08:00:02.855ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
3382025-11-13T08:00:02.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
3392025-11-13T08:00:02.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
3402025-11-13T08:00:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 217.548517ms iteration = 3 status = {"errors":[],"finish_invoked_ok":[]}
3412025-11-13T08:00:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 217.642829ms iteration = 3 status = {"errors":[],"garbage_collect_requested":[]}
3422025-11-13T08:00:02.870ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3432025-11-13T08:00:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3442025-11-13T08:00:02.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3452025-11-13T08:00:02.881ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3462025-11-13T08:00:02.882ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6011, tv_nsec: 234784631 } })
3472025-11-13T08:00:02.882ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
3482025-11-13T08:00:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
3492025-11-13T08:00:02.884ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3502025-11-13T08:00:02.885ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
3512025-11-13T08:00:02.885ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
3522025-11-13T08:00:02.885ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 2
3532025-11-13T08:00:02.885ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 278.831002ms iteration = 2 status = {"generation":2}
3542025-11-13T08:00:02.885ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 4 reason = Dependency
3552025-11-13T08:00:02.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3562025-11-13T08:00:02.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3572025-11-13T08:00:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3582025-11-13T08:00:02.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
3592025-11-13T08:00:02.889ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
3602025-11-13T08:00:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
3612025-11-13T08:00:02.896ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
3622025-11-13T08:00:02.896ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 248.570537ms iteration = 5 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
3632025-11-13T08:00:02.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3642025-11-13T08:00:02.903ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3652025-11-13T08:00:02.903ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
3662025-11-13T08:00:02.904ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3672025-11-13T08:00:02.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3682025-11-13T08:00:02.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
3692025-11-13T08:00:02.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3702025-11-13T08:00:02.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
3712025-11-13T08:00:02.907ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
3722025-11-13T08:00:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
3732025-11-13T08:00:02.907ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 300.155651ms iteration = 3 status = {}
3742025-11-13T08:00:02.908ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
3752025-11-13T08:00:02.908ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 290.258016ms iteration = 4 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
3762025-11-13T08:00:02.909ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = DeployNexusRecords step id = Ensure
3772025-11-13T08:00:02.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3782025-11-13T08:00:02.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3792025-11-13T08:00:02.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3802025-11-13T08:00:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
3812025-11-13T08:00:02.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3822025-11-13T08:00:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3832025-11-13T08:00:02.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
3842025-11-13T08:00:02.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
3852025-11-13T08:00:02.920ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3862025-11-13T08:00:02.921ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
3872025-11-13T08:00:02.921ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 278.325582ms iteration = 3 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
3882025-11-13T08:00:02.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3892025-11-13T08:00:02.921ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
3902025-11-13T08:00:02.921ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
3912025-11-13T08:00:02.921ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
3922025-11-13T08:00:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
3932025-11-13T08:00:02.924ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
3942025-11-13T08:00:02.924ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
3952025-11-13T08:00:02.924ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
3962025-11-13T08:00:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
3972025-11-13T08:00:02.927ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
3982025-11-13T08:00:02.928ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
3992025-11-13T08:00:02.931ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
4002025-11-13T08:00:02.931ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 283.580055ms iteration = 5 status = "NoSitrep"
4012025-11-13T08:00:02.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
4022025-11-13T08:00:02.932ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
4032025-11-13T08:00:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
4042025-11-13T08:00:02.932ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
4052025-11-13T08:00:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
4062025-11-13T08:00:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
4072025-11-13T08:00:02.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
4082025-11-13T08:00:02.935ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
4092025-11-13T08:00:02.935ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4102025-11-13T08:00:02.935ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
4112025-11-13T08:00:02.935ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
4122025-11-13T08:00:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
4132025-11-13T08:00:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
4142025-11-13T08:00:02.936ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
4152025-11-13T08:00:02.936ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
4162025-11-13T08:00:02.936ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 288.95543ms iteration = 3 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
4172025-11-13T08:00:02.937ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9ce1295-6cad-4f2a-b29a-65eaf4c35705", "content-length": "81", "date": "Thu, 13 Nov 2025 08:00:02 GMT"} })
4182025-11-13T08:00:02.937ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
4192025-11-13T08:00:02.937ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
4202025-11-13T08:00:02.937ZINFOnexus (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal file = nexus/src/app/background/tasks/dns_config.rs:87 generation = 1
4212025-11-13T08:00:02.937ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 330.392915ms iteration = 2 status = {"generation":1}
4222025-11-13T08:00:02.937ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 4 reason = Dependency
4232025-11-13T08:00:02.940ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 297.851755ms iteration = 7 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
4242025-11-13T08:00:02.975ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
4252025-11-13T08:00:02.975ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
4262025-11-13T08:00:02.976ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb981a60-4127-41c4-a8c8-b5b653307ef8", "content-length": "81", "date": "Thu, 13 Nov 2025 08:00:02 GMT"} })
4272025-11-13T08:00:03.001ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
4282025-11-13T08:00:03.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
4292025-11-13T08:00:03.005ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:77
4302025-11-13T08:00:03.005ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 397.949004ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
4312025-11-13T08:00:03.005ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 357.411566ms iteration = 3 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
4322025-11-13T08:00:03.005ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
4332025-11-13T08:00:03.005ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 387.217292ms iteration = 6 status = {"physical_disks_added":0}
4342025-11-13T08:00:03.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
4352025-11-13T08:00:03.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
4362025-11-13T08:00:03.023ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
4372025-11-13T08:00:03.023ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
4382025-11-13T08:00:03.024ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "63754f8c-c3b2-448d-9402-73ca9c8338f4", "content-length": "26", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
4392025-11-13T08:00:03.024ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
4402025-11-13T08:00:03.025ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = SledAgent step id = Ensure
4412025-11-13T08:00:03.026ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
4422025-11-13T08:00:03.039ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
4432025-11-13T08:00:03.039ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
4442025-11-13T08:00:03.040ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "fde3d6d3-20b3-4531-b635-62263e670eb6", "content-length": "81", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
4452025-11-13T08:00:03.040ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
4462025-11-13T08:00:03.040ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
4472025-11-13T08:00:03.040ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 397.908182ms iteration = 3 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
4482025-11-13T08:00:03.043ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 396.034616ms iteration = 3 status = {"disabled":false,"errors":[],"sps":[]}
4492025-11-13T08:00:03.050ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4502025-11-13T08:00:03.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4512025-11-13T08:00:03.066ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
4522025-11-13T08:00:03.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
4532025-11-13T08:00:03.070ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
4542025-11-13T08:00:03.071ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = / versions = all
4552025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /access versions = all
4562025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /assets/{path} versions = all
4572025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/auth versions = all
4582025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/confirm versions = all
4592025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/success versions = all
4602025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /device/token versions = all
4612025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /device/verify versions = all
4622025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes versions = all
4632025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/probes versions = all
4642025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/probes/{probe} versions = all
4652025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/probes/{probe} versions = all
4662025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles versions = all
4672025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /experimental/v1/system/support-bundles versions = all
4682025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
4692025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
4702025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
4712025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
4722025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
4732025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
4742025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
4752025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
4762025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /images versions = all
4772025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login versions = all
4782025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/local versions = all
4792025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
4802025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
4812025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
4822025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /lookup/{path} versions = all
4832025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects/{path} versions = all
4842025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /projects-new versions = all
4852025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Groups versions = all
4862025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /scim/v2/Groups versions = all
4872025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /scim/v2/Groups/{group_id} versions = all
4882025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Groups/{group_id} versions = all
4892025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PATCH path = /scim/v2/Groups/{group_id} versions = all
4902025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /scim/v2/Groups/{group_id} versions = all
4912025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Users versions = all
4922025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /scim/v2/Users versions = all
4932025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /scim/v2/Users/{user_id} versions = all
4942025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /scim/v2/Users/{user_id} versions = all
4952025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PATCH path = /scim/v2/Users/{user_id} versions = all
4962025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /scim/v2/Users/{user_id} versions = all
4972025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /settings/{path} versions = all
4982025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /system/{path} versions = all
4992025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /utilization versions = all
5002025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups versions = all
5012025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups versions = all
5022025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
5032025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
5042025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
5052025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
5062025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
5072025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
5082025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
5092025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-classes versions = all
5102025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers versions = all
5112025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
5122025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver} versions = all
5132025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
5142025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
5152025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
5162025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
5172025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/alerts/{alert_id}/resend versions = all
5182025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups versions = all
5192025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups versions = all
5202025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
5212025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
5222025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
5232025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
5242025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
5252025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
5262025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
5272025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/auth-settings versions = all
5282025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/auth-settings versions = all
5292025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates versions = all
5302025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/certificates versions = all
5312025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/certificates/{certificate} versions = all
5322025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/certificates/{certificate} versions = all
5332025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks versions = all
5342025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks versions = all
5352025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/disks/{disk} versions = all
5362025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/disks/{disk} versions = all
5372025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write versions = all
5382025-11-13T08:00:03.072ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
5392025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
5402025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/disks/{disk}/finalize versions = all
5412025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips versions = all
5422025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips versions = all
5432025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
5442025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/floating-ips/{floating_ip} versions = all
5452025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
5462025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
5472025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
5482025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups versions = all
5492025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/groups/{group_id} versions = all
5502025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images versions = all
5512025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images versions = all
5522025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/images/{image} versions = all
5532025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/images/{image} versions = all
5542025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/demote versions = all
5552025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/images/{image}/promote versions = all
5562025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances versions = all
5572025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances versions = all
5582025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance} versions = all
5592025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance} versions = all
5602025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/instances/{instance} versions = all
5612025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
5622025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
5632025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/disks versions = all
5642025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/attach versions = all
5652025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/disks/detach versions = all
5662025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/external-ips versions = all
5672025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
5682025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
5692025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/reboot versions = all
5702025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console versions = all
5712025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
5722025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
5732025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/start versions = all
5742025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/instances/{instance}/stop versions = all
5752025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-addresses versions = all
5762025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-addresses versions = all
5772025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
5782025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateway-ip-pools versions = all
5792025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateway-ip-pools versions = all
5802025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
5812025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways versions = all
5822025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/internet-gateways versions = all
5832025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
5842025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/internet-gateways/{gateway} versions = all
5852025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools versions = all
5862025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ip-pools/{pool} versions = all
5872025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/login/{silo_name}/local versions = all
5882025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/logout versions = all
5892025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me versions = all
5902025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/access-tokens versions = all
5912025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
5922025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/groups versions = all
5932025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys versions = all
5942025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/me/ssh-keys versions = all
5952025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
5962025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
5972025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/metrics/{metric_name} versions = all
5982025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces versions = all
5992025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/network-interfaces versions = all
6002025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/network-interfaces/{interface} versions = all
6012025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/network-interfaces/{interface} versions = all
6022025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/network-interfaces/{interface} versions = all
6032025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/ping versions = all
6042025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/policy versions = all
6052025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/policy versions = all
6062025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects versions = all
6072025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/projects versions = all
6082025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/projects/{project} versions = all
6092025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project} versions = all
6102025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project} versions = all
6112025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/projects/{project}/policy versions = all
6122025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/projects/{project}/policy versions = all
6132025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots versions = all
6142025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/snapshots versions = all
6152025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/snapshots/{snapshot} versions = all
6162025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/snapshots/{snapshot} versions = all
6172025-11-13T08:00:03.073ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/audit-log versions = all
6182025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks versions = all
6192025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
6202025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
6212025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks versions = all
6222025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
6232025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds versions = all
6242025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/sleds versions = all
6252025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
6262025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
6272025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
6282025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
6292025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
6302025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port versions = all
6312025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
6322025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
6332025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
6342025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
6352025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
6362025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches versions = all
6372025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
6382025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers versions = all
6392025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users versions = all
6402025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
6412025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
6422025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/identity-providers/saml versions = all
6432025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
6442025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools versions = all
6452025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools versions = all
6462025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
6472025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool} versions = all
6482025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool} versions = all
6492025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
6502025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
6512025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
6522025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
6532025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
6542025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
6552025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
6562025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
6572025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service versions = all
6582025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/ip-pools-service/ranges versions = all
6592025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
6602025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
6612025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/metrics/{metric_name} versions = all
6622025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot versions = all
6632025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/address-lot versions = all
6642025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
6652025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
6662025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
6672025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/allow-list versions = all
6682025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/allow-list versions = all
6692025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-disable versions = all
6702025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bfd-enable versions = all
6712025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bfd-status versions = all
6722025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp versions = all
6732025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp versions = all
6742025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/bgp versions = all
6752025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set versions = all
6762025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
6772025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
6782025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
6792025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-exported versions = all
6802025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-message-history versions = all
6812025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
6822025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/bgp-status versions = all
6832025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/inbound-icmp versions = all
6842025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/networking/inbound-icmp versions = all
6852025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/loopback-address versions = all
6862025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/loopback-address versions = all
6872025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
6882025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
6892025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings versions = all
6902025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/networking/switch-port-settings versions = all
6912025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
6922025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/policy versions = all
6932025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/policy versions = all
6942025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/scim/tokens versions = all
6952025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/scim/tokens versions = all
6962025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/scim/tokens/{token_id} versions = all
6972025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/scim/tokens/{token_id} versions = all
6982025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silo-quotas versions = all
6992025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos versions = all
7002025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/silos versions = all
7012025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/silos/{silo} versions = all
7022025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo} versions = all
7032025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
7042025-11-13T08:00:03.074ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/policy versions = all
7052025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/policy versions = all
7062025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/silos/{silo}/quotas versions = all
7072025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
7082025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/timeseries/query versions = all
7092025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/timeseries/schemas versions = all
7102025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/repositories versions = all
7112025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/repositories versions = all
7122025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/repositories/{system_version} versions = all
7132025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/status versions = all
7142025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/system/update/target-release versions = all
7152025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/trust-roots versions = all
7162025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/system/update/trust-roots versions = all
7172025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
7182025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
7192025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users versions = all
7202025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users/{user_id} versions = all
7212025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin versions = all
7222025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/users-builtin/{user} versions = all
7232025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos versions = all
7242025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/system/utilization/silos/{silo} versions = all
7252025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/timeseries/query versions = all
7262025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users versions = all
7272025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id} versions = all
7282025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id}/access-tokens versions = all
7292025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/users/{user_id}/logout versions = all
7302025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/users/{user_id}/sessions versions = all
7312025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/utilization versions = all
7322025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-firewall-rules versions = all
7332025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-firewall-rules versions = all
7342025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes versions = all
7352025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-router-routes versions = all
7362025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
7372025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-router-routes/{route} versions = all
7382025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-router-routes/{route} versions = all
7392025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers versions = all
7402025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-routers versions = all
7412025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-routers/{router} versions = all
7422025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-routers/{router} versions = all
7432025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-routers/{router} versions = all
7442025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets versions = all
7452025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpc-subnets versions = all
7462025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
7472025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet} versions = all
7482025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
7492025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
7502025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs versions = all
7512025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/vpcs versions = all
7522025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/vpcs/{vpc} versions = all
7532025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/vpcs/{vpc} versions = all
7542025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/vpcs/{vpc} versions = all
7552025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-receivers versions = all
7562025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
7572025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = GET path = /v1/webhook-secrets versions = all
7582025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = POST path = /v1/webhook-secrets versions = all
7592025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): registered endpoint local_addr = 172.30.2.5:443 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
7602025-11-13T08:00:03.075ZINFOnexus (dropshot_external): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = 172.30.2.5:443
7612025-11-13T08:00:03.075ZDEBGnexus (dropshot_external): successfully registered DTrace USDT probes local_addr = 172.30.2.5:443
7622025-11-13T08:00:03.075ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7632025-11-13T08:00:03.075ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7642025-11-13T08:00:03.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7652025-11-13T08:00:03.076ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
7662025-11-13T08:00:03.079ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
7672025-11-13T08:00:03.079ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7682025-11-13T08:00:03.080ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
7692025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = / versions = all
7702025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /access versions = all
7712025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /assets/{path} versions = all
7722025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/auth versions = all
7732025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/confirm versions = all
7742025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/success versions = all
7752025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /device/token versions = all
7762025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /device/verify versions = all
7772025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes versions = all
7782025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/probes versions = all
7792025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
7802025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
7812025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
7822025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
7832025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
7842025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
7852025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /experimental/v1/system/support-bundles/{bundle_id} versions = all
7862025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
7872025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download versions = all
7882025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
7892025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{bundle_id}/download/{file} versions = all
7902025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /experimental/v1/system/support-bundles/{bundle_id}/index versions = all
7912025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /images versions = all
7922025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login versions = all
7932025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/local versions = all
7942025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
7952025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
7962025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
7972025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /lookup/{path} versions = all
7982025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects/{path} versions = all
7992025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /projects-new versions = all
8002025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Groups versions = all
8012025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /scim/v2/Groups versions = all
8022025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /scim/v2/Groups/{group_id} versions = all
8032025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Groups/{group_id} versions = all
8042025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PATCH path = /scim/v2/Groups/{group_id} versions = all
8052025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /scim/v2/Groups/{group_id} versions = all
8062025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Users versions = all
8072025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /scim/v2/Users versions = all
8082025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /scim/v2/Users/{user_id} versions = all
8092025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /scim/v2/Users/{user_id} versions = all
8102025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PATCH path = /scim/v2/Users/{user_id} versions = all
8112025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /scim/v2/Users/{user_id} versions = all
8122025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /settings/{path} versions = all
8132025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /system/{path} versions = all
8142025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /utilization versions = all
8152025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups versions = all
8162025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups versions = all
8172025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
8182025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
8192025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
8202025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
8212025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
8222025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
8232025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
8242025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-classes versions = all
8252025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers versions = all
8262025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver} versions = all
8272025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver} versions = all
8282025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/alert-receivers/{receiver}/deliveries versions = all
8292025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/probe versions = all
8302025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alert-receivers/{receiver}/subscriptions versions = all
8312025-11-13T08:00:03.088ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/alert-receivers/{receiver}/subscriptions/{subscription} versions = all
8322025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/alerts/{alert_id}/resend versions = all
8332025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups versions = all
8342025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups versions = all
8352025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
8362025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
8372025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
8382025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
8392025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
8402025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
8412025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
8422025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/auth-settings versions = all
8432025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/auth-settings versions = all
8442025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates versions = all
8452025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/certificates versions = all
8462025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
8472025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/certificates/{certificate} versions = all
8482025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks versions = all
8492025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks versions = all
8502025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/disks/{disk} versions = all
8512025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/disks/{disk} versions = all
8522025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
8532025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
8542025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
8552025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
8562025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips versions = all
8572025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips versions = all
8582025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
8592025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
8602025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
8612025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
8622025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
8632025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups versions = all
8642025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/groups/{group_id} versions = all
8652025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images versions = all
8662025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images versions = all
8672025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/images/{image} versions = all
8682025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/images/{image} versions = all
8692025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/demote versions = all
8702025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/images/{image}/promote versions = all
8712025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances versions = all
8722025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances versions = all
8732025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance} versions = all
8742025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance} versions = all
8752025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/instances/{instance} versions = all
8762025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
8772025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
8782025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
8792025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
8802025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
8812025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
8822025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
8832025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
8842025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
8852025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
8862025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
8872025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
8882025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/start versions = all
8892025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
8902025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
8912025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
8922025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
8932025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
8942025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
8952025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
8962025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways versions = all
8972025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/internet-gateways versions = all
8982025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
8992025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
9002025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools versions = all
9012025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
9022025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
9032025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/logout versions = all
9042025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me versions = all
9052025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/access-tokens versions = all
9062025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/me/access-tokens/{token_id} versions = all
9072025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/groups versions = all
9082025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys versions = all
9092025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/me/ssh-keys versions = all
9102025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
9112025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
9122025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
9132025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces versions = all
9142025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/network-interfaces versions = all
9152025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
9162025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
9172025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
9182025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/ping versions = all
9192025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/policy versions = all
9202025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/policy versions = all
9212025-11-13T08:00:03.089ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects versions = all
9222025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/projects versions = all
9232025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/projects/{project} versions = all
9242025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project} versions = all
9252025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project} versions = all
9262025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/projects/{project}/policy versions = all
9272025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
9282025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots versions = all
9292025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/snapshots versions = all
9302025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
9312025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
9322025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/audit-log versions = all
9332025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks versions = all
9342025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
9352025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
9362025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks versions = all
9372025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
9382025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds versions = all
9392025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/sleds versions = all
9402025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
9412025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
9422025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
9432025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
9442025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
9452025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
9462025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
9472025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
9482025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
9492025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
9502025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
9512025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches versions = all
9522025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
9532025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers versions = all
9542025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
9552025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
9562025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
9572025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
9582025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
9592025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools versions = all
9602025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools versions = all
9612025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
9622025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
9632025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
9642025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
9652025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
9662025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
9672025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
9682025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
9692025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
9702025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
9712025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
9722025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service versions = all
9732025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
9742025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
9752025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
9762025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
9772025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot versions = all
9782025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/address-lot versions = all
9792025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
9802025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot} versions = all
9812025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
9822025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/allow-list versions = all
9832025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
9842025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
9852025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
9862025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
9872025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
9882025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp versions = all
9892025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/bgp versions = all
9902025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
9912025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
9922025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
9932025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
9942025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
9952025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
9962025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
9972025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
9982025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/inbound-icmp versions = all
9992025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/networking/inbound-icmp versions = all
10002025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
10012025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
10022025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
10032025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
10042025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
10052025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
10062025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
10072025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/policy versions = all
10082025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/policy versions = all
10092025-11-13T08:00:03.090ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/scim/tokens versions = all
10102025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/scim/tokens versions = all
10112025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/scim/tokens/{token_id} versions = all
10122025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/scim/tokens/{token_id} versions = all
10132025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silo-quotas versions = all
10142025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos versions = all
10152025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/silos versions = all
10162025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
10172025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo} versions = all
10182025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
10192025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
10202025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
10212025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
10222025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
10232025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/timeseries/query versions = all
10242025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
10252025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/repositories versions = all
10262025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/repositories versions = all
10272025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/repositories/{system_version} versions = all
10282025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/status versions = all
10292025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/system/update/target-release versions = all
10302025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/trust-roots versions = all
10312025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/system/update/trust-roots versions = all
10322025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/system/update/trust-roots/{trust_root_id} versions = all
10332025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/update/trust-roots/{trust_root_id} versions = all
10342025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users versions = all
10352025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users/{user_id} versions = all
10362025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin versions = all
10372025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
10382025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos versions = all
10392025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
10402025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/timeseries/query versions = all
10412025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users versions = all
10422025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id} versions = all
10432025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id}/access-tokens versions = all
10442025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/users/{user_id}/logout versions = all
10452025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/users/{user_id}/sessions versions = all
10462025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/utilization versions = all
10472025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
10482025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
10492025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes versions = all
10502025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-router-routes versions = all
10512025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
10522025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
10532025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
10542025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers versions = all
10552025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-routers versions = all
10562025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
10572025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
10582025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
10592025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets versions = all
10602025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpc-subnets versions = all
10612025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
10622025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
10632025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
10642025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
10652025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs versions = all
10662025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/vpcs versions = all
10672025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
10682025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
10692025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
10702025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-receivers versions = all
10712025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = PUT path = /v1/webhook-receivers/{receiver} versions = all
10722025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = GET path = /v1/webhook-secrets versions = all
10732025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = POST path = /v1/webhook-secrets versions = all
10742025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): registered endpoint local_addr = [fd00:1122:3344:101::a]:12228 method = DELETE path = /v1/webhook-secrets/{secret_id} versions = all
10752025-11-13T08:00:03.091ZINFOnexus (dropshot_external_techport): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:12228
10762025-11-13T08:00:03.091ZDEBGnexus (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:12228
10772025-11-13T08:00:03.094ZDEBGnexus (nexus-producer-server): registered DTrace probes
10782025-11-13T08:00:03.101ZDEBGnexus (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::a]:32941 method = GET path = /{producer_id} versions = all
10792025-11-13T08:00:03.101ZINFOnexus (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:328 local_addr = [fd00:1122:3344:101::a]:32941
10802025-11-13T08:00:03.101ZDEBGnexus (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::a]:32941
10812025-11-13T08:00:03.101ZDEBGnexus (nexus-producer-server): Requested any available port, Dropshot server has been bound to [fd00:1122:3344:101::a]:32941
10822025-11-13T08:00:03.101ZDEBGnexus (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [fd00:1122:3344:101::a]:12221
10832025-11-13T08:00:03.101ZDEBGnexus (nexus-producer-server): starting producer registration task
10842025-11-13T08:00:03.102ZINFOnexus (nexus-producer-server): starting oximeter metric producer server address = [fd00:1122:3344:101::a]:32941 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = bcb58129-79cc-406c-aa16-455a5cd9f031
10852025-11-13T08:00:03.102ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
10862025-11-13T08:00:03.102ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
10872025-11-13T08:00:03.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10882025-11-13T08:00:03.107ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
10892025-11-13T08:00:03.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10902025-11-13T08:00:03.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10912025-11-13T08:00:03.108ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
10922025-11-13T08:00:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
10932025-11-13T08:00:03.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10942025-11-13T08:00:03.114ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10952025-11-13T08:00:03.115ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
10962025-11-13T08:00:03.116ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
10972025-11-13T08:00:03.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
10982025-11-13T08:00:03.117ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
10992025-11-13T08:00:03.117ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11002025-11-13T08:00:03.117ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
11012025-11-13T08:00:03.117ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
11022025-11-13T08:00:03.117ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f47e054f-1b72-4586-8950-8e61e636d4a7", "content-length": "372", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
11032025-11-13T08:00:03.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11042025-11-13T08:00:03.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
11052025-11-13T08:00:03.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11062025-11-13T08:00:03.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11072025-11-13T08:00:03.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11082025-11-13T08:00:03.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11092025-11-13T08:00:03.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11102025-11-13T08:00:03.144ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
11112025-11-13T08:00:03.144ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
11122025-11-13T08:00:03.144ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "17320779-71b3-4420-a111-ed39cd1ca9fd", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
11132025-11-13T08:00:03.144ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11142025-11-13T08:00:03.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11152025-11-13T08:00:03.146ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
11162025-11-13T08:00:03.149ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11172025-11-13T08:00:03.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11182025-11-13T08:00:03.153ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
11192025-11-13T08:00:03.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
11202025-11-13T08:00:03.158ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
11212025-11-13T08:00:03.159ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
11222025-11-13T08:00:03.159ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "57788468-9e9c-4501-9d29-39b0f8dbd296", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
11232025-11-13T08:00:03.161ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 518.347361ms iteration = 4 status = {}
11242025-11-13T08:00:03.196ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
11252025-11-13T08:00:03.197ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ae552fce-07b3-40f8-b63c-7c57a9c1f27a", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11262025-11-13T08:00:03.197ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11272025-11-13T08:00:03.199ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "db9ca54b-6e5e-4561-936f-e4fe93924cce", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11282025-11-13T08:00:03.199ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11292025-11-13T08:00:03.202ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
11302025-11-13T08:00:03.202ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 316.86676ms iteration = 4 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
11312025-11-13T08:00:03.206ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
11322025-11-13T08:00:03.208ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "5e5cf642-3f1c-469c-860d-607eef55352a", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
11332025-11-13T08:00:03.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11342025-11-13T08:00:03.211ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = FirewallRules step id = Ensure
11352025-11-13T08:00:03.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11362025-11-13T08:00:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11372025-11-13T08:00:03.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11382025-11-13T08:00:03.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11392025-11-13T08:00:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
11402025-11-13T08:00:03.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11412025-11-13T08:00:03.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11422025-11-13T08:00:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11432025-11-13T08:00:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11442025-11-13T08:00:03.218ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6011, tv_nsec: 571094293 } })
11452025-11-13T08:00:03.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11462025-11-13T08:00:03.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11472025-11-13T08:00:03.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11482025-11-13T08:00:03.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11492025-11-13T08:00:03.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11502025-11-13T08:00:03.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11512025-11-13T08:00:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11522025-11-13T08:00:03.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11532025-11-13T08:00:03.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11542025-11-13T08:00:03.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11552025-11-13T08:00:03.227ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
11562025-11-13T08:00:03.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11572025-11-13T08:00:03.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11582025-11-13T08:00:03.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11592025-11-13T08:00:03.235ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
11602025-11-13T08:00:03.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11612025-11-13T08:00:03.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11622025-11-13T08:00:03.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11632025-11-13T08:00:03.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11642025-11-13T08:00:03.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11652025-11-13T08:00:03.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11662025-11-13T08:00:03.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11672025-11-13T08:00:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11682025-11-13T08:00:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11692025-11-13T08:00:03.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11702025-11-13T08:00:03.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11712025-11-13T08:00:03.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11722025-11-13T08:00:03.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11732025-11-13T08:00:03.245ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
11742025-11-13T08:00:03.245ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 602.608767ms iteration = 2 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
11752025-11-13T08:00:03.246ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
11762025-11-13T08:00:03.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11772025-11-13T08:00:03.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11782025-11-13T08:00:03.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11792025-11-13T08:00:03.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11802025-11-13T08:00:03.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11812025-11-13T08:00:03.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11822025-11-13T08:00:03.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11832025-11-13T08:00:03.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11842025-11-13T08:00:03.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11852025-11-13T08:00:03.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11862025-11-13T08:00:03.257ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
11872025-11-13T08:00:03.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11882025-11-13T08:00:03.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11892025-11-13T08:00:03.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11902025-11-13T08:00:03.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11912025-11-13T08:00:03.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
11922025-11-13T08:00:03.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
11932025-11-13T08:00:03.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11942025-11-13T08:00:03.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11952025-11-13T08:00:03.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11962025-11-13T08:00:03.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
11972025-11-13T08:00:03.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
11982025-11-13T08:00:03.265ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
11992025-11-13T08:00:03.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12002025-11-13T08:00:03.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12012025-11-13T08:00:03.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12022025-11-13T08:00:03.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12032025-11-13T08:00:03.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12042025-11-13T08:00:03.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12052025-11-13T08:00:03.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12062025-11-13T08:00:03.273ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
12072025-11-13T08:00:03.273ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:64546
12082025-11-13T08:00:03.276ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:64546 req_id = 06412ad6-8599-408a-ba54-7c024c398aaa uri = /metrics/producers version = 1.0.0
12092025-11-13T08:00:03.279ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12102025-11-13T08:00:03.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12112025-11-13T08:00:03.279ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12122025-11-13T08:00:03.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12132025-11-13T08:00:03.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12142025-11-13T08:00:03.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12152025-11-13T08:00:03.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12162025-11-13T08:00:03.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12172025-11-13T08:00:03.286ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
12182025-11-13T08:00:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12192025-11-13T08:00:03.287ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "3eb63709-2c99-4f14-9c35-06e62327bf1b", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
12202025-11-13T08:00:03.287ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
12212025-11-13T08:00:03.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12222025-11-13T08:00:03.290ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
12232025-11-13T08:00:03.290ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 641.789777ms iteration = 2 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
12242025-11-13T08:00:03.290ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 5 reason = Dependency
12252025-11-13T08:00:03.290ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
12262025-11-13T08:00:03.290ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12272025-11-13T08:00:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12282025-11-13T08:00:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12292025-11-13T08:00:03.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12302025-11-13T08:00:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12312025-11-13T08:00:03.292ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12322025-11-13T08:00:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12332025-11-13T08:00:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12342025-11-13T08:00:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12352025-11-13T08:00:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12362025-11-13T08:00:03.296ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12372025-11-13T08:00:03.296ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12382025-11-13T08:00:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12392025-11-13T08:00:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12402025-11-13T08:00:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12412025-11-13T08:00:03.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12422025-11-13T08:00:03.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12432025-11-13T08:00:03.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12442025-11-13T08:00:03.304ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
12452025-11-13T08:00:03.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
12462025-11-13T08:00:03.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
12472025-11-13T08:00:03.305ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:64546 req_id = 06412ad6-8599-408a-ba54-7c024c398aaa roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
12482025-11-13T08:00:03.305ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:64546 req_id = 06412ad6-8599-408a-ba54-7c024c398aaa resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
12492025-11-13T08:00:03.306ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
12502025-11-13T08:00:03.306ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12512025-11-13T08:00:03.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12522025-11-13T08:00:03.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12532025-11-13T08:00:03.307ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
12542025-11-13T08:00:03.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12552025-11-13T08:00:03.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12562025-11-13T08:00:03.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12572025-11-13T08:00:03.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12582025-11-13T08:00:03.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12592025-11-13T08:00:03.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12602025-11-13T08:00:03.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12612025-11-13T08:00:03.316ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12622025-11-13T08:00:03.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12632025-11-13T08:00:03.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12642025-11-13T08:00:03.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12652025-11-13T08:00:03.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12662025-11-13T08:00:03.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12672025-11-13T08:00:03.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12682025-11-13T08:00:03.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12692025-11-13T08:00:03.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12702025-11-13T08:00:03.325ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
12712025-11-13T08:00:03.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12722025-11-13T08:00:03.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12732025-11-13T08:00:03.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12742025-11-13T08:00:03.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12752025-11-13T08:00:03.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12762025-11-13T08:00:03.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12772025-11-13T08:00:03.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12782025-11-13T08:00:03.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12792025-11-13T08:00:03.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12802025-11-13T08:00:03.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12812025-11-13T08:00:03.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12822025-11-13T08:00:03.336ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
12832025-11-13T08:00:03.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12842025-11-13T08:00:03.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12852025-11-13T08:00:03.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12862025-11-13T08:00:03.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12872025-11-13T08:00:03.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12882025-11-13T08:00:03.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12892025-11-13T08:00:03.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12902025-11-13T08:00:03.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
12912025-11-13T08:00:03.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
12922025-11-13T08:00:03.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12932025-11-13T08:00:03.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12942025-11-13T08:00:03.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12952025-11-13T08:00:03.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12962025-11-13T08:00:03.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12972025-11-13T08:00:03.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
12982025-11-13T08:00:03.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
12992025-11-13T08:00:03.348ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
13002025-11-13T08:00:03.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13012025-11-13T08:00:03.354ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13022025-11-13T08:00:03.354ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13032025-11-13T08:00:03.355ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13042025-11-13T08:00:03.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13052025-11-13T08:00:03.362ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
13062025-11-13T08:00:03.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13072025-11-13T08:00:03.362ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "94616026-ad7a-402b-9b5c-83853b94b8fa", "content-length": "691", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
13082025-11-13T08:00:03.362ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
13092025-11-13T08:00:03.362ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
13102025-11-13T08:00:03.362ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
13112025-11-13T08:00:03.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13122025-11-13T08:00:03.364ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 722.445833ms iteration = 3 status = {}
13132025-11-13T08:00:03.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13142025-11-13T08:00:03.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13152025-11-13T08:00:03.373ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13162025-11-13T08:00:03.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13172025-11-13T08:00:03.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13182025-11-13T08:00:03.386ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
13192025-11-13T08:00:03.386ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13202025-11-13T08:00:03.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13212025-11-13T08:00:03.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13222025-11-13T08:00:03.393ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
13232025-11-13T08:00:03.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
13242025-11-13T08:00:03.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
13252025-11-13T08:00:03.396ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
13262025-11-13T08:00:03.396ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
13272025-11-13T08:00:03.397ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
13282025-11-13T08:00:03.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13292025-11-13T08:00:03.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13302025-11-13T08:00:03.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13312025-11-13T08:00:03.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13322025-11-13T08:00:03.413ZINFOnexus: assigned collector to new producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d file = nexus/src/app/oximeter.rs:76 producer_id = bcb58129-79cc-406c-aa16-455a5cd9f031
13332025-11-13T08:00:03.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13342025-11-13T08:00:03.417ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 140939 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:64546 req_id = 06412ad6-8599-408a-ba54-7c024c398aaa response_code = 201 uri = /metrics/producers
13352025-11-13T08:00:03.417ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "06412ad6-8599-408a-ba54-7c024c398aaa", "content-length": "41", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
13362025-11-13T08:00:03.419ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
13372025-11-13T08:00:03.420ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
13382025-11-13T08:00:03.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13392025-11-13T08:00:03.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13402025-11-13T08:00:03.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13412025-11-13T08:00:03.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13422025-11-13T08:00:03.426ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
13432025-11-13T08:00:03.426ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13442025-11-13T08:00:03.426ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ce4b16a-affe-443f-a620-c60977162fca", "content-length": "372", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
13452025-11-13T08:00:03.426ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13462025-11-13T08:00:03.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13472025-11-13T08:00:03.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13482025-11-13T08:00:03.429ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
13492025-11-13T08:00:03.430ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "deb4df44-961b-49ab-9d80-a71e3cf46fed", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
13502025-11-13T08:00:03.430ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
13512025-11-13T08:00:03.431ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
13522025-11-13T08:00:03.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13532025-11-13T08:00:03.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13542025-11-13T08:00:03.432ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "afdc97be-1edb-44dc-9c39-f516182757b5", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
13552025-11-13T08:00:03.432ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
13562025-11-13T08:00:03.434ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "d02cb887-9f34-4ecc-90ed-712e46d7921e", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
13572025-11-13T08:00:03.434ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
13582025-11-13T08:00:03.436ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
13592025-11-13T08:00:03.436ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 499.228593ms iteration = 4 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
13602025-11-13T08:00:03.443ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
13612025-11-13T08:00:03.443ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
13622025-11-13T08:00:03.444ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9e762440-445d-4dfe-8ac9-eb36886d4529", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
13632025-11-13T08:00:03.444ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13642025-11-13T08:00:03.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13652025-11-13T08:00:03.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13662025-11-13T08:00:03.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13672025-11-13T08:00:03.447ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13682025-11-13T08:00:03.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13692025-11-13T08:00:03.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13702025-11-13T08:00:03.450ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13712025-11-13T08:00:03.450ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13722025-11-13T08:00:03.451ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
13732025-11-13T08:00:03.451ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
13742025-11-13T08:00:03.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
13752025-11-13T08:00:03.453ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13762025-11-13T08:00:03.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13772025-11-13T08:00:03.454ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
13782025-11-13T08:00:03.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13792025-11-13T08:00:03.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13802025-11-13T08:00:03.456ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
13812025-11-13T08:00:03.456ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
13822025-11-13T08:00:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13832025-11-13T08:00:03.457ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "0e62ec06-f9c8-4aa6-8690-ca27a13ca340", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
13842025-11-13T08:00:03.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13852025-11-13T08:00:03.458ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 167.758237ms iteration = 5 status = {}
13862025-11-13T08:00:03.459ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13872025-11-13T08:00:03.462ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
13882025-11-13T08:00:03.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13892025-11-13T08:00:03.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13902025-11-13T08:00:03.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
13912025-11-13T08:00:03.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
13922025-11-13T08:00:03.470ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
13932025-11-13T08:00:03.470ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
13942025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
13952025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
13962025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13972025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13982025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
13992025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14002025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
14012025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
14022025-11-13T08:00:03.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14032025-11-13T08:00:03.471ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
14042025-11-13T08:00:03.471ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
14052025-11-13T08:00:03.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
14062025-11-13T08:00:03.474ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
14072025-11-13T08:00:03.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14082025-11-13T08:00:03.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14092025-11-13T08:00:03.474ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
14102025-11-13T08:00:03.474ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
14112025-11-13T08:00:03.476ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
14122025-11-13T08:00:03.476ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
14132025-11-13T08:00:03.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14142025-11-13T08:00:03.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14152025-11-13T08:00:03.511ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14162025-11-13T08:00:03.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14172025-11-13T08:00:03.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14182025-11-13T08:00:03.518ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
14192025-11-13T08:00:03.518ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
14202025-11-13T08:00:03.519ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
14212025-11-13T08:00:03.519ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
14222025-11-13T08:00:03.519ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = b6c5aa7b-c14c-424e-9c04-838c177b6b9b parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
14232025-11-13T08:00:03.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14242025-11-13T08:00:03.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14252025-11-13T08:00:03.520ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c34299f9-1b71-4a35-a6c2-3c1ff148943d", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
14262025-11-13T08:00:03.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14272025-11-13T08:00:03.521ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
14282025-11-13T08:00:03.521ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = Dns step id = Ensure
14292025-11-13T08:00:03.521ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
14302025-11-13T08:00:03.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14312025-11-13T08:00:03.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14322025-11-13T08:00:03.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14332025-11-13T08:00:03.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14342025-11-13T08:00:03.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14352025-11-13T08:00:03.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
14362025-11-13T08:00:03.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
14372025-11-13T08:00:03.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
14382025-11-13T08:00:03.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14392025-11-13T08:00:03.528ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
14402025-11-13T08:00:03.528ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
14412025-11-13T08:00:03.528ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 909.801346ms iteration = 7 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
14422025-11-13T08:00:03.564ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
14432025-11-13T08:00:03.564ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
14442025-11-13T08:00:03.564ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
14452025-11-13T08:00:03.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14462025-11-13T08:00:03.565ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14472025-11-13T08:00:03.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14482025-11-13T08:00:03.571ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14492025-11-13T08:00:03.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14502025-11-13T08:00:03.571ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14512025-11-13T08:00:03.610ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
14522025-11-13T08:00:03.610ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
14532025-11-13T08:00:03.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14542025-11-13T08:00:03.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14552025-11-13T08:00:03.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14562025-11-13T08:00:03.615ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14572025-11-13T08:00:03.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14582025-11-13T08:00:03.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14592025-11-13T08:00:03.618ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
14602025-11-13T08:00:03.618ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
14612025-11-13T08:00:03.618ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
14622025-11-13T08:00:03.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14632025-11-13T08:00:03.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14642025-11-13T08:00:03.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14652025-11-13T08:00:03.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14662025-11-13T08:00:03.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14672025-11-13T08:00:03.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14682025-11-13T08:00:03.625ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
14692025-11-13T08:00:03.625ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
14702025-11-13T08:00:03.625ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
14712025-11-13T08:00:03.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14722025-11-13T08:00:03.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14732025-11-13T08:00:03.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742025-11-13T08:00:03.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14752025-11-13T08:00:03.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14762025-11-13T08:00:03.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14772025-11-13T08:00:03.632ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
14782025-11-13T08:00:03.632ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
14792025-11-13T08:00:03.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14802025-11-13T08:00:03.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14812025-11-13T08:00:03.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14822025-11-13T08:00:03.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14832025-11-13T08:00:03.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14842025-11-13T08:00:03.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14852025-11-13T08:00:03.639ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
14862025-11-13T08:00:03.639ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
14872025-11-13T08:00:03.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14882025-11-13T08:00:03.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14892025-11-13T08:00:03.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14902025-11-13T08:00:03.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
14912025-11-13T08:00:03.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14922025-11-13T08:00:03.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14932025-11-13T08:00:03.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14942025-11-13T08:00:03.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14952025-11-13T08:00:03.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
14962025-11-13T08:00:03.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
14972025-11-13T08:00:03.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14982025-11-13T08:00:03.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
14992025-11-13T08:00:03.652ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
15002025-11-13T08:00:03.652ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
15012025-11-13T08:00:03.652ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = OmicronZones step id = Cleanup
15022025-11-13T08:00:03.652ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = OmicronZones step id = Cleanup
15032025-11-13T08:00:03.652ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = PhysicalDisks step id = Cleanup
15042025-11-13T08:00:03.652ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = Clickhouse step id = Ensure
15052025-11-13T08:00:03.652ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = Clickhouse step id = Ensure
15062025-11-13T08:00:03.684ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
15072025-11-13T08:00:03.709ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "e6aafa1f-0db7-4858-b094-1bdeeeaaeb93", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
15082025-11-13T08:00:03.709ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
15092025-11-13T08:00:03.709ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
15102025-11-13T08:00:03.716ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "426eb4f7-9a18-42f0-ab00-60d53809c715", "content-length": "682", "date": "Thu, 13 Nov 2025 08:00:03 GMT"} })
15112025-11-13T08:00:03.716ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15122025-11-13T08:00:03.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15132025-11-13T08:00:03.719ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15142025-11-13T08:00:03.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15152025-11-13T08:00:03.723ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15162025-11-13T08:00:03.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15172025-11-13T08:00:03.725ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
15182025-11-13T08:00:03.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
15192025-11-13T08:00:03.732ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.089174242s iteration = 3 status = {}
15202025-11-13T08:00:04.079ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "05260d89-75ab-4553-b203-236aab1b9823", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15212025-11-13T08:00:04.081ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = SupportBundles step id = Cleanup
15222025-11-13T08:00:04.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15232025-11-13T08:00:04.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15242025-11-13T08:00:04.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15252025-11-13T08:00:04.084ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
15262025-11-13T08:00:04.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15272025-11-13T08:00:04.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15282025-11-13T08:00:04.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15292025-11-13T08:00:04.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15302025-11-13T08:00:04.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15312025-11-13T08:00:04.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
15322025-11-13T08:00:04.136ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = OmicronZones step id = Cleanup
15332025-11-13T08:00:04.136ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
15342025-11-13T08:00:04.136ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
15352025-11-13T08:00:04.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15362025-11-13T08:00:04.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15372025-11-13T08:00:04.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15382025-11-13T08:00:04.139ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
15392025-11-13T08:00:04.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15402025-11-13T08:00:04.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15412025-11-13T08:00:04.175ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
15422025-11-13T08:00:04.175ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
15432025-11-13T08:00:04.175ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = Cockroach step id = Ensure
15442025-11-13T08:00:04.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
15452025-11-13T08:00:04.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
15462025-11-13T08:00:04.202ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
15472025-11-13T08:00:04.202ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 5c4dc1dd-cbec-4735-ac44-12d3318e2704 step component = MgsUpdates step id = Ensure
15482025-11-13T08:00:04.202ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 5 reason = Dependency
15492025-11-13T08:00:04.202ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15502025-11-13T08:00:04.202ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Dependency
15512025-11-13T08:00:04.202ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15522025-11-13T08:00:04.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15532025-11-13T08:00:04.202ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.594892934s iteration = 3 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":2940,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":237553661,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":237553611,"secs":0}},"event_index":1,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":237566201,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":64335275,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":64335235,"secs":0}},"event_index":2,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":301920556,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":115957780,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":115957730,"secs":0}},"event_index":3,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":417900076,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":185217132,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":185217082,"secs":0}},"event_index":4,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":603139638,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":310761309,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":310761269,"secs":0}},"event_index":5,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":913914947,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":130849232,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":130849202,"secs":0}},"event_index":6,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":44772249,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4740,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4700,"secs":0}},"event_index":7,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":44786440,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2410,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2370,"secs":0}},"event_index":8,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":44791220,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3350,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3310,"secs":0}},"event_index":9,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":44796310,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1820,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1780,"secs":0}},"event_index":10,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":44799660,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":428279600,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":428279550,"secs":0}},"event_index":11,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":473080140,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":55148934,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":55148904,"secs":0}},"event_index":12,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":528239204,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39462866,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":39462806,"secs":0}},"event_index":13,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":567715530,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":26777906,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":26777866,"secs":0}},"event_index":14,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":594503317,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5830,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5790,"secs":0}},"event_index":15,"execution_id":"99d7996d-b705-40e7-b0ae-f4a84ac22599","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":594517177,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
15542025-11-13T08:00:04.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15552025-11-13T08:00:04.204ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15562025-11-13T08:00:04.204ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15572025-11-13T08:00:04.206ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
15582025-11-13T08:00:04.206ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
15592025-11-13T08:00:04.206ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15602025-11-13T08:00:04.206ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15612025-11-13T08:00:04.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
15622025-11-13T08:00:04.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15632025-11-13T08:00:04.246ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15642025-11-13T08:00:04.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15652025-11-13T08:00:04.248ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15662025-11-13T08:00:04.250ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
15672025-11-13T08:00:04.250ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
15682025-11-13T08:00:04.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
15692025-11-13T08:00:04.252ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
15702025-11-13T08:00:04.252ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 49.747038ms iteration = 5 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
15712025-11-13T08:00:04.355ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(f853806d-24be-431a-8043-0ee3214f07c9, 0), (3e46c821-345a-483e-b369-5d7323fe389a, 0), (badd5a66-f8f5-414c-9527-1e9f4eb27e70, 0)]
15722025-11-13T08:00:04.355ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6012, tv_nsec: 708109275 } })
15732025-11-13T08:00:04.388ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6012, tv_nsec: 740667225 } })
15742025-11-13T08:00:04.388ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
15752025-11-13T08:00:04.388ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
15762025-11-13T08:00:04.388ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
15772025-11-13T08:00:04.389ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "68b32a5f-e94e-41be-bb27-f9c91ec9cca3", "content-length": "476", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15782025-11-13T08:00:04.389ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
15792025-11-13T08:00:04.389ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e261a28-a94c-4af8-9bb7-95216af4773f", "content-length": "734", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15802025-11-13T08:00:04.389ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
15812025-11-13T08:00:04.390ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bff131b-cc2d-45ec-ba91-5ccc87ba19dd", "content-length": "111", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15822025-11-13T08:00:04.390ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
15832025-11-13T08:00:04.390ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8787f2ee-8a75-4ed0-86e6-c731a883fd4d", "content-length": "111", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15842025-11-13T08:00:04.390ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
15852025-11-13T08:00:04.390ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b801e033-72d7-41aa-91b9-de5cb5a80513", "content-length": "163", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15862025-11-13T08:00:04.390ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
15872025-11-13T08:00:04.391ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "efc2d335-7cfc-45cf-a044-0a12996876e9", "content-length": "163", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15882025-11-13T08:00:04.391ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
15892025-11-13T08:00:04.391ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "506a1e29-bad7-4c93-9384-065f7cf9a8e0", "content-length": "166", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15902025-11-13T08:00:04.391ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
15912025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cb70037-415d-4c93-b411-4cf311e6dc4f", "content-length": "166", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15922025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
15932025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "72e0b165-033b-4395-a6df-72c489c1f4b2", "content-length": "702", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15942025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
15952025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa9dda59-69a6-4916-9d5c-dd476f1e8c96", "content-length": "727", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15962025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
15972025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "829eaa03-7f6d-4827-a50a-ce632ccf6e41", "content-length": "729", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
15982025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
15992025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2620519-09f2-4c19-bdad-2997f3b77d7d", "content-length": "728", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16002025-11-13T08:00:04.392ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
16012025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43fc6100-cf72-4d16-871d-298fa1420ad1", "content-length": "722", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16022025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
16032025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f520b64-5192-4ac8-9d21-8ef430ba93e6", "content-length": "10", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16042025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
16052025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da02a5f5-54a5-4438-94a7-52e3f5b68058", "content-length": "144", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16062025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
16072025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b0ca315-bbb7-4060-aa6a-066c71b7b2d5", "content-length": "147", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16082025-11-13T08:00:04.393ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
16092025-11-13T08:00:04.394ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b0f1811-25ac-4792-87e4-32010db63d6d", "content-length": "109", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16102025-11-13T08:00:04.394ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
16112025-11-13T08:00:04.394ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a54a290e-f04f-4d50-ae36-8aa259b99c8e", "content-length": "109", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16122025-11-13T08:00:04.394ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
16132025-11-13T08:00:04.394ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c9ddc76-107f-469e-b6a2-25cb033f79a8", "content-length": "163", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16142025-11-13T08:00:04.394ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
16152025-11-13T08:00:04.395ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c035a0c6-fe4e-4c23-b8fa-8393b92e86f1", "content-length": "163", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16162025-11-13T08:00:04.395ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
16172025-11-13T08:00:04.395ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e259d425-e13c-41e8-87ea-1a3f19d9c487", "content-length": "166", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16182025-11-13T08:00:04.395ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
16192025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d32d4094-744f-4b1c-8467-c84bd891f687", "content-length": "166", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16202025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
16212025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "471fed81-ee1f-4f47-9a8c-b2afbd394be4", "content-length": "702", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16222025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16232025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ccd67cf-ec2d-4b7e-b819-6227047e528e", "content-length": "727", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16242025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16252025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92dba937-f77c-47d6-a97a-8b14189bfcef", "content-length": "729", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16262025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
16272025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3062bcc-c76c-4c6d-a09f-df3f7bcbd297", "content-length": "728", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16282025-11-13T08:00:04.396ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16292025-11-13T08:00:04.397ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16302025-11-13T08:00:04.398ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16312025-11-13T08:00:04.400ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16322025-11-13T08:00:04.400ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16332025-11-13T08:00:04.400ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16342025-11-13T08:00:04.402ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16352025-11-13T08:00:04.404ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
16362025-11-13T08:00:04.404ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16372025-11-13T08:00:04.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16382025-11-13T08:00:04.512ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
16392025-11-13T08:00:04.512ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
16402025-11-13T08:00:04.592ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e177bb2-5266-4470-af6f-c763d39ab58f", "content-length": "95175", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16412025-11-13T08:00:04.594ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16422025-11-13T08:00:04.594ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
16432025-11-13T08:00:04.594ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
16442025-11-13T08:00:04.594ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
16452025-11-13T08:00:04.594ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
16462025-11-13T08:00:04.594ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
16472025-11-13T08:00:04.594ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
16482025-11-13T08:00:04.594ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
16492025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ced5123-3f34-4747-b1f9-b3d1112bda29", "content-length": "64", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16502025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bfc640c-226f-446f-9ff7-57ed26e651dd", "content-length": "64", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16512025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
16522025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
16532025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c34b3db-fbb0-42be-b271-400c0a20ba4d", "content-length": "64", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16542025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "98aa85bd-276d-4f73-99c1-137803f11485", "content-length": "64", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16552025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b19aa91-0bcf-4171-be8a-5bc4030d6371", "content-length": "64", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16562025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
16572025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
16582025-11-13T08:00:04.595ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
16592025-11-13T08:00:04.634ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8db51ee5-e9bd-40c7-880a-dfaa9efb1cd6", "content-length": "371699", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16602025-11-13T08:00:04.634ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d64f4fd5-9cc8-4ac9-826e-249494939704", "content-length": "373038", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16612025-11-13T08:00:04.636ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bf7856d-3eb7-4b76-95da-4f0f3a2803c6", "content-length": "404787", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16622025-11-13T08:00:04.637ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc0edca9-922f-4286-b0c1-a8a02e9f9558", "content-length": "382982", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16632025-11-13T08:00:04.642ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "757990d9-2ac3-46e2-b428-080afb3e728e", "content-length": "365786", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16642025-11-13T08:00:04.644ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
16652025-11-13T08:00:04.644ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
16662025-11-13T08:00:04.644ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
16672025-11-13T08:00:04.644ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
16682025-11-13T08:00:04.644ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
16692025-11-13T08:00:04.680ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
16702025-11-13T08:00:04.680ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
16712025-11-13T08:00:04.684ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "e287cd65-ed9f-4910-989d-87ed388d1ceb", "content-length": "125", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16722025-11-13T08:00:04.685ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
16732025-11-13T08:00:04.775ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
16742025-11-13T08:00:04.775ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
16752025-11-13T08:00:04.776ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e4447fd-ea9e-4628-83ed-58107da7ab00", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16762025-11-13T08:00:04.776ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
16772025-11-13T08:00:04.778ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
16782025-11-13T08:00:04.778ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
16792025-11-13T08:00:04.779ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d60de9d3-5b61-44b3-b0b2-8749fb8f0ec6", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16802025-11-13T08:00:04.779ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
16812025-11-13T08:00:04.780ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
16822025-11-13T08:00:04.780ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
16832025-11-13T08:00:04.781ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3546d1ce-3d60-4ff2-8ce7-74c42a6f0b63", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:00:04 GMT"} })
16842025-11-13T08:00:04.781ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
16852025-11-13T08:00:04.782ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
16862025-11-13T08:00:04.782ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
16872025-11-13T08:00:04.783ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16882025-11-13T08:00:04.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16892025-11-13T08:00:04.786ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16902025-11-13T08:00:04.788ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
16912025-11-13T08:00:04.788ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
16922025-11-13T08:00:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
16932025-11-13T08:00:05.005ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 07399175-748d-4999-b636-5f12bc3d81dd file = nexus/db-queries/src/db/datastore/inventory.rs:1613
16942025-11-13T08:00:05.005ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 07399175-748d-4999-b636-5f12bc3d81dd time_started = 2025-11-13 08:00:04.388861 UTC
16952025-11-13T08:00:05.005ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 8 reason = Dependency
16962025-11-13T08:00:05.005ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
16972025-11-13T08:00:05.005ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 803.048216ms iteration = 3 status = {"collection_id":"07399175-748d-4999-b636-5f12bc3d81dd","time_done":"2025-11-13 08:00:04.782338 UTC","time_started":"2025-11-13 08:00:04.388861 UTC"}
16982025-11-13T08:00:05.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
16992025-11-13T08:00:05.007ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17002025-11-13T08:00:05.010ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
17012025-11-13T08:00:05.010ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
17022025-11-13T08:00:05.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
17032025-11-13T08:00:05.013ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
17042025-11-13T08:00:05.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
17052025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 72.500965ms iteration = 8 status = {"Loaded":{"collection_id":"6ebc3749-17f1-41eb-93de-ab321c41fd59","time_loaded":"2025-11-13T08:00:05.005536946Z","time_started":"2025-11-13T08:00:04.446770Z"}}
17062025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 8 reason = Dependency
17072025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 7 reason = Dependency
17082025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 5 reason = Dependency
17092025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17102025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17112025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17122025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17132025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17142025-11-13T08:00:05.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17152025-11-13T08:00:05.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17162025-11-13T08:00:05.079ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17172025-11-13T08:00:05.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17182025-11-13T08:00:05.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17192025-11-13T08:00:05.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17202025-11-13T08:00:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17212025-11-13T08:00:05.083ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17222025-11-13T08:00:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17232025-11-13T08:00:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17242025-11-13T08:00:05.083ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17252025-11-13T08:00:05.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17262025-11-13T08:00:05.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17272025-11-13T08:00:05.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17282025-11-13T08:00:05.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17292025-11-13T08:00:05.085ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17302025-11-13T08:00:05.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17312025-11-13T08:00:05.087ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17322025-11-13T08:00:05.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17332025-11-13T08:00:05.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17342025-11-13T08:00:05.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17352025-11-13T08:00:05.087ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17362025-11-13T08:00:05.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17372025-11-13T08:00:05.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17382025-11-13T08:00:05.089ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
17392025-11-13T08:00:05.089ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
17402025-11-13T08:00:05.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17412025-11-13T08:00:05.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
17422025-11-13T08:00:05.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17432025-11-13T08:00:05.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17442025-11-13T08:00:05.093ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17452025-11-13T08:00:05.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17462025-11-13T08:00:05.093ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17472025-11-13T08:00:05.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17482025-11-13T08:00:05.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17492025-11-13T08:00:05.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17502025-11-13T08:00:05.096ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
17512025-11-13T08:00:05.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17522025-11-13T08:00:05.096ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17532025-11-13T08:00:05.096ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17542025-11-13T08:00:05.098ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17552025-11-13T08:00:05.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17562025-11-13T08:00:05.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17572025-11-13T08:00:05.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17582025-11-13T08:00:05.102ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17592025-11-13T08:00:05.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17602025-11-13T08:00:05.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17612025-11-13T08:00:05.106ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
17622025-11-13T08:00:05.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17632025-11-13T08:00:05.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17642025-11-13T08:00:05.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17652025-11-13T08:00:05.110ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17662025-11-13T08:00:05.110ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17672025-11-13T08:00:05.110ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17682025-11-13T08:00:05.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17692025-11-13T08:00:05.113ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
17702025-11-13T08:00:05.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
17712025-11-13T08:00:05.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
17722025-11-13T08:00:05.115ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
17732025-11-13T08:00:05.115ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 37.885554ms iteration = 5 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"6ebc3749-17f1-41eb-93de-ab321c41fd59","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
17742025-11-13T08:00:05.140ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 61.958838ms iteration = 7 status = {"physical_disks_added":0}
17752025-11-13T08:00:05.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17762025-11-13T08:00:05.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17772025-11-13T08:00:05.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17782025-11-13T08:00:05.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17792025-11-13T08:00:05.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17802025-11-13T08:00:05.151ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
17812025-11-13T08:00:05.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17822025-11-13T08:00:05.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17832025-11-13T08:00:05.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17842025-11-13T08:00:05.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17852025-11-13T08:00:05.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17862025-11-13T08:00:05.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
17872025-11-13T08:00:05.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17882025-11-13T08:00:05.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17892025-11-13T08:00:05.159ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17902025-11-13T08:00:05.162ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
17912025-11-13T08:00:05.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17922025-11-13T08:00:05.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17932025-11-13T08:00:05.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17942025-11-13T08:00:05.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17952025-11-13T08:00:05.165ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17962025-11-13T08:00:05.168ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
17972025-11-13T08:00:05.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
17982025-11-13T08:00:05.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
17992025-11-13T08:00:05.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18002025-11-13T08:00:05.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18012025-11-13T08:00:05.171ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18022025-11-13T08:00:05.174ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
18032025-11-13T08:00:05.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18042025-11-13T08:00:05.174ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18052025-11-13T08:00:05.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18062025-11-13T08:00:05.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18072025-11-13T08:00:05.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18082025-11-13T08:00:05.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
18092025-11-13T08:00:05.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18102025-11-13T08:00:05.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18112025-11-13T08:00:05.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18122025-11-13T08:00:05.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18132025-11-13T08:00:05.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18142025-11-13T08:00:05.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18152025-11-13T08:00:05.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18162025-11-13T08:00:05.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18172025-11-13T08:00:05.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18182025-11-13T08:00:05.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18192025-11-13T08:00:05.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18202025-11-13T08:00:05.212ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18212025-11-13T08:00:05.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18222025-11-13T08:00:05.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18232025-11-13T08:00:05.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18242025-11-13T08:00:05.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18252025-11-13T08:00:05.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18262025-11-13T08:00:05.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18272025-11-13T08:00:05.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18282025-11-13T08:00:05.223ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18292025-11-13T08:00:05.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18302025-11-13T08:00:05.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18312025-11-13T08:00:05.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18322025-11-13T08:00:05.227ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
18332025-11-13T08:00:05.227ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18342025-11-13T08:00:05.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18352025-11-13T08:00:05.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18362025-11-13T08:00:05.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18372025-11-13T08:00:05.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18382025-11-13T08:00:05.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18392025-11-13T08:00:05.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18402025-11-13T08:00:05.236ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
18412025-11-13T08:00:05.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18422025-11-13T08:00:05.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18432025-11-13T08:00:05.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18442025-11-13T08:00:05.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18452025-11-13T08:00:05.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18462025-11-13T08:00:05.241ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
18472025-11-13T08:00:05.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18482025-11-13T08:00:05.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18492025-11-13T08:00:05.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18502025-11-13T08:00:05.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18512025-11-13T08:00:05.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18522025-11-13T08:00:05.255ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
18532025-11-13T08:00:05.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18542025-11-13T08:00:05.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18552025-11-13T08:00:05.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18562025-11-13T08:00:05.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18572025-11-13T08:00:05.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18582025-11-13T08:00:05.261ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
18592025-11-13T08:00:05.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18602025-11-13T08:00:05.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18612025-11-13T08:00:05.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18622025-11-13T08:00:05.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18632025-11-13T08:00:05.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18642025-11-13T08:00:05.268ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
18652025-11-13T08:00:05.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18662025-11-13T08:00:05.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18672025-11-13T08:00:05.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18682025-11-13T08:00:05.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18692025-11-13T08:00:05.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18702025-11-13T08:00:05.274ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
18712025-11-13T08:00:05.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18722025-11-13T08:00:05.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18732025-11-13T08:00:05.312ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
18742025-11-13T08:00:05.312ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
18752025-11-13T08:00:05.312ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 5cf73f7e-06bb-4b3b-b145-dae009c4664d parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
18762025-11-13T08:00:05.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18772025-11-13T08:00:05.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18782025-11-13T08:00:05.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18792025-11-13T08:00:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
18802025-11-13T08:00:05.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
18812025-11-13T08:00:05.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
18822025-11-13T08:00:05.320ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
18832025-11-13T08:00:05.320ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
18842025-11-13T08:00:05.320ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 242.461107ms iteration = 8 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
18852025-11-13T08:00:07.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 14 reason = Timeout
18862025-11-13T08:00:07.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18872025-11-13T08:00:07.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18882025-11-13T08:00:07.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18892025-11-13T08:00:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
18902025-11-13T08:00:07.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18912025-11-13T08:00:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
18922025-11-13T08:00:07.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
18932025-11-13T08:00:07.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.389086ms iteration = 14 status = {"config_updated":false}
18942025-11-13T08:00:12.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 8 reason = Timeout
18952025-11-13T08:00:12.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 15 reason = Timeout
18962025-11-13T08:00:12.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
18972025-11-13T08:00:12.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
18982025-11-13T08:00:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
18992025-11-13T08:00:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19002025-11-13T08:00:12.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19012025-11-13T08:00:12.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19022025-11-13T08:00:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
19032025-11-13T08:00:12.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19042025-11-13T08:00:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19052025-11-13T08:00:12.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19062025-11-13T08:00:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19072025-11-13T08:00:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19082025-11-13T08:00:12.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
19092025-11-13T08:00:12.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.710592ms iteration = 15 status = {"config_updated":false}
19102025-11-13T08:00:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19112025-11-13T08:00:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19122025-11-13T08:00:12.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19132025-11-13T08:00:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
19142025-11-13T08:00:12.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19152025-11-13T08:00:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19162025-11-13T08:00:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 8 reason = Timeout
19172025-11-13T08:00:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19182025-11-13T08:00:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19192025-11-13T08:00:12.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19202025-11-13T08:00:12.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
19212025-11-13T08:00:12.645ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
19222025-11-13T08:00:12.645ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.200394ms iteration = 8 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
19232025-11-13T08:00:12.677ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.390466ms iteration = 8 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
19242025-11-13T08:00:15.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 208e1f51-dbea-40bd-9222-84903f7623d5 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
19252025-11-13T08:00:15.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 208e1f51-dbea-40bd-9222-84903f7623d5 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
19262025-11-13T08:00:15.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 208e1f51-dbea-40bd-9222-84903f7623d5 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
19272025-11-13T08:00:15.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3557 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 208e1f51-dbea-40bd-9222-84903f7623d5 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
19282025-11-13T08:00:15.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4c6138be-9313-4157-916c-9980b3ed8f7a uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
19292025-11-13T08:00:15.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4c6138be-9313-4157-916c-9980b3ed8f7a roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
19302025-11-13T08:00:15.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4c6138be-9313-4157-916c-9980b3ed8f7a resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
19312025-11-13T08:00:15.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 1834 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4c6138be-9313-4157-916c-9980b3ed8f7a response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
19322025-11-13T08:00:15.622ZINFOnexus (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:32941 remote_addr = [fd00:1122:3344:101::d]:52304
19332025-11-13T08:00:15.642ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 19689 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 22ef2d59-d0c2-44c7-ae98-4191ee51e4e7 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
19342025-11-13T08:00:17.617ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 9 reason = Timeout
19352025-11-13T08:00:17.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 16 reason = Timeout
19362025-11-13T08:00:17.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19372025-11-13T08:00:17.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19382025-11-13T08:00:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19392025-11-13T08:00:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19402025-11-13T08:00:17.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19412025-11-13T08:00:17.619ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19422025-11-13T08:00:17.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
19432025-11-13T08:00:17.623ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
19442025-11-13T08:00:17.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
19452025-11-13T08:00:17.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19462025-11-13T08:00:17.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
19472025-11-13T08:00:17.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19482025-11-13T08:00:17.625ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6ebc3749-17f1-41eb-93de-ab321c41fd59 original_time_started = 2025-11-13 08:00:04.446770 UTC
19492025-11-13T08:00:17.625ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.587449ms iteration = 9 status = {"Loaded":{"collection_id":"6ebc3749-17f1-41eb-93de-ab321c41fd59","time_loaded":"2025-11-13T08:00:17.617847214Z","time_started":"2025-11-13T08:00:04.446770Z"}}
19502025-11-13T08:00:17.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
19512025-11-13T08:00:17.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.948567ms iteration = 16 status = {"config_updated":false}
19522025-11-13T08:00:17.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 6 reason = Timeout
19532025-11-13T08:00:17.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19542025-11-13T08:00:17.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 6 reason = Timeout
19552025-11-13T08:00:17.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
19562025-11-13T08:00:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19572025-11-13T08:00:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
19582025-11-13T08:00:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19592025-11-13T08:00:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19602025-11-13T08:00:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19612025-11-13T08:00:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19622025-11-13T08:00:17.651ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
19632025-11-13T08:00:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
19642025-11-13T08:00:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
19652025-11-13T08:00:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
19662025-11-13T08:00:17.653ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.59905ms iteration = 6 status = "NoSitrep"
19672025-11-13T08:00:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19682025-11-13T08:00:17.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19692025-11-13T08:00:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
19702025-11-13T08:00:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
19712025-11-13T08:00:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19722025-11-13T08:00:17.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
19732025-11-13T08:00:17.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
19742025-11-13T08:00:17.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.597259ms iteration = 6 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
19752025-11-13T08:00:22.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 9 reason = Timeout
19762025-11-13T08:00:22.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 17 reason = Timeout
19772025-11-13T08:00:22.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19782025-11-13T08:00:22.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19792025-11-13T08:00:22.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19802025-11-13T08:00:22.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19812025-11-13T08:00:22.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19822025-11-13T08:00:22.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19832025-11-13T08:00:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
19842025-11-13T08:00:22.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19852025-11-13T08:00:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
19862025-11-13T08:00:22.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
19872025-11-13T08:00:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19882025-11-13T08:00:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
19892025-11-13T08:00:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19902025-11-13T08:00:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19912025-11-13T08:00:22.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
19922025-11-13T08:00:22.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.775503ms iteration = 17 status = {"config_updated":false}
19932025-11-13T08:00:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19942025-11-13T08:00:22.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
19952025-11-13T08:00:22.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
19962025-11-13T08:00:22.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
19972025-11-13T08:00:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 9 reason = Timeout
19982025-11-13T08:00:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
19992025-11-13T08:00:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20002025-11-13T08:00:22.644ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
20012025-11-13T08:00:22.644ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.9498ms iteration = 9 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
20022025-11-13T08:00:22.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
20032025-11-13T08:00:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
20042025-11-13T08:00:22.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.533159ms iteration = 9 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
20052025-11-13T08:00:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 313 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 78adc7a5-b2d5-4659-9848-3d9e45d7ab94 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
20062025-11-13T08:00:27.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 18 reason = Timeout
20072025-11-13T08:00:27.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
20082025-11-13T08:00:27.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
20092025-11-13T08:00:27.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20102025-11-13T08:00:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
20112025-11-13T08:00:27.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
20122025-11-13T08:00:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
20132025-11-13T08:00:27.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
20142025-11-13T08:00:27.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.712902ms iteration = 18 status = {"config_updated":false}
20152025-11-13T08:00:30.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4d0e298c-0274-4f0b-9a05-f33789d3bc96 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
20162025-11-13T08:00:30.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4d0e298c-0274-4f0b-9a05-f33789d3bc96 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
20172025-11-13T08:00:30.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4d0e298c-0274-4f0b-9a05-f33789d3bc96 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
20182025-11-13T08:00:30.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3466 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4d0e298c-0274-4f0b-9a05-f33789d3bc96 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
20192025-11-13T08:00:30.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4fb4014c-625f-4350-b3e5-deccb64ac0f6 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
20202025-11-13T08:00:30.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4fb4014c-625f-4350-b3e5-deccb64ac0f6 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
20212025-11-13T08:00:30.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4fb4014c-625f-4350-b3e5-deccb64ac0f6 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
20222025-11-13T08:00:30.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2546 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 4fb4014c-625f-4350-b3e5-deccb64ac0f6 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
20232025-11-13T08:00:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 4 reason = Timeout
20242025-11-13T08:00:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 4 reason = Timeout
20252025-11-13T08:00:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
20262025-11-13T08:00:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 4 reason = Timeout
20272025-11-13T08:00:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
20282025-11-13T08:00:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
20292025-11-13T08:00:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
20302025-11-13T08:00:32.613ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6040, tv_nsec: 965540474 } })
20312025-11-13T08:00:32.613ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
20322025-11-13T08:00:32.615ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6040, tv_nsec: 967543123 } })
20332025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 6 reason = Timeout
20342025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 10 reason = Timeout
20352025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 8 reason = Timeout
20362025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20372025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
20382025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20392025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 10 reason = Timeout
20402025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 19 reason = Timeout
20412025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20422025-11-13T08:00:32.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
20432025-11-13T08:00:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
20442025-11-13T08:00:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20452025-11-13T08:00:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20462025-11-13T08:00:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20472025-11-13T08:00:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
20482025-11-13T08:00:32.622ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20492025-11-13T08:00:32.622ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20502025-11-13T08:00:32.622ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20512025-11-13T08:00:32.623ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20522025-11-13T08:00:32.624ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.064106ms iteration = 4 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
20532025-11-13T08:00:32.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20542025-11-13T08:00:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20552025-11-13T08:00:32.627ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20562025-11-13T08:00:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20572025-11-13T08:00:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
20582025-11-13T08:00:32.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
20592025-11-13T08:00:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
20602025-11-13T08:00:32.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20612025-11-13T08:00:32.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20622025-11-13T08:00:32.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
20632025-11-13T08:00:32.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
20642025-11-13T08:00:32.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20652025-11-13T08:00:32.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
20662025-11-13T08:00:32.630ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20672025-11-13T08:00:32.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20682025-11-13T08:00:32.631ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6ebc3749-17f1-41eb-93de-ab321c41fd59 original_time_started = 2025-11-13 08:00:04.446770 UTC
20692025-11-13T08:00:32.631ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20702025-11-13T08:00:32.631ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.013666ms iteration = 10 status = {"Loaded":{"collection_id":"6ebc3749-17f1-41eb-93de-ab321c41fd59","time_loaded":"2025-11-13T08:00:32.618618976Z","time_started":"2025-11-13T08:00:04.446770Z"}}
20712025-11-13T08:00:32.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20722025-11-13T08:00:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
20732025-11-13T08:00:32.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20742025-11-13T08:00:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20752025-11-13T08:00:32.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
20762025-11-13T08:00:32.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.100967ms iteration = 19 status = {"config_updated":false}
20772025-11-13T08:00:32.632ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20782025-11-13T08:00:32.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20792025-11-13T08:00:32.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
20802025-11-13T08:00:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
20812025-11-13T08:00:32.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20822025-11-13T08:00:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
20832025-11-13T08:00:32.637ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
20842025-11-13T08:00:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
20852025-11-13T08:00:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
20862025-11-13T08:00:32.638ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
20872025-11-13T08:00:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
20882025-11-13T08:00:32.639ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
20892025-11-13T08:00:32.639ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 20.730847ms iteration = 6 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
20902025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 4 reason = Timeout
20912025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 6 reason = Timeout
20922025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
20932025-11-13T08:00:32.641ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
20942025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 4 reason = Timeout
20952025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
20962025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
20972025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 4 reason = Timeout
20982025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
20992025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 10 reason = Timeout
21002025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21012025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 4 reason = Timeout
21022025-11-13T08:00:32.641ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21032025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 4 reason = Timeout
21042025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21052025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 4 reason = Timeout
21062025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
21072025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21082025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21092025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21102025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21112025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
21122025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21132025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21142025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
21152025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
21162025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
21172025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
21182025-11-13T08:00:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
21192025-11-13T08:00:32.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21202025-11-13T08:00:32.645ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21212025-11-13T08:00:32.645ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21222025-11-13T08:00:32.645ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21232025-11-13T08:00:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21242025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 4 reason = Timeout
21252025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 4 reason = Timeout
21262025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21272025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 7 reason = Timeout
21282025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
21292025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
21302025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 7 reason = Timeout
21312025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
21322025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 4 reason = Timeout
21332025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 4 reason = Timeout
21342025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
21352025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 4 reason = Timeout
21362025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
21372025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 4 reason = Timeout
21382025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
21392025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
21402025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
21412025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
21422025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21432025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
21442025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
21452025-11-13T08:00:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
21462025-11-13T08:00:32.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6041, tv_nsec: 1212835 } })
21472025-11-13T08:00:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
21482025-11-13T08:00:32.649ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
21492025-11-13T08:00:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
21502025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
21512025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21522025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21532025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
21542025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
21552025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
21562025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21572025-11-13T08:00:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
21582025-11-13T08:00:32.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21592025-11-13T08:00:32.651ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21602025-11-13T08:00:32.655ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
21612025-11-13T08:00:32.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
21622025-11-13T08:00:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
21632025-11-13T08:00:32.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
21642025-11-13T08:00:32.657ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
21652025-11-13T08:00:32.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
21662025-11-13T08:00:32.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21672025-11-13T08:00:32.663ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 15.869732ms iteration = 7 status = "NoSitrep"
21682025-11-13T08:00:32.665ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 23.718496ms iteration = 4 status = 0
21692025-11-13T08:00:32.665ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6041, tv_nsec: 18155528 } })
21702025-11-13T08:00:32.666ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
21712025-11-13T08:00:32.667ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
21722025-11-13T08:00:32.667ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
21732025-11-13T08:00:32.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
21742025-11-13T08:00:32.669ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
21752025-11-13T08:00:32.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
21762025-11-13T08:00:32.691ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
21772025-11-13T08:00:32.691ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 73.010875ms iteration = 10 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
21782025-11-13T08:00:32.722ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
21792025-11-13T08:00:32.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
21802025-11-13T08:00:32.724ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 82.525203ms iteration = 10 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
21812025-11-13T08:00:32.731ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 112.599984ms iteration = 8 status = {"physical_disks_added":0}
21822025-11-13T08:00:32.734ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
21832025-11-13T08:00:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
21842025-11-13T08:00:32.739ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
21852025-11-13T08:00:32.739ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
21862025-11-13T08:00:32.739ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
21872025-11-13T08:00:32.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
21882025-11-13T08:00:32.741ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 92.898687ms iteration = 4 status = {"errors":[],"finish_invoked_ok":[]}
21892025-11-13T08:00:32.742ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 94.812655ms iteration = 4 status = {"errors":[],"garbage_collect_requested":[]}
21902025-11-13T08:00:32.743ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
21912025-11-13T08:00:32.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
21922025-11-13T08:00:32.747ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21932025-11-13T08:00:32.749ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
21942025-11-13T08:00:32.750ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 108.110205ms iteration = 4 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
21952025-11-13T08:00:32.750ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
21962025-11-13T08:00:32.753ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 104.984244ms iteration = 4 status = {"errors":[],"requests_created_ok":[]}
21972025-11-13T08:00:32.755ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
21982025-11-13T08:00:32.755ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
21992025-11-13T08:00:32.755ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22002025-11-13T08:00:32.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22012025-11-13T08:00:32.755ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eeb8c13-15f4-40e8-abe9-9d2ec404d1b2", "content-length": "26", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22022025-11-13T08:00:32.755ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22032025-11-13T08:00:32.756ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
22042025-11-13T08:00:32.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
22052025-11-13T08:00:32.757ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
22062025-11-13T08:00:32.759ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
22072025-11-13T08:00:32.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
22082025-11-13T08:00:32.764ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
22092025-11-13T08:00:32.766ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 158.487676ms iteration = 4 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
22102025-11-13T08:00:32.786ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
22112025-11-13T08:00:32.787ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "219e6fec-abca-48cd-b9b7-d5b3a5eb170f", "content-length": "78", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22122025-11-13T08:00:32.787ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
22132025-11-13T08:00:32.787ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
22142025-11-13T08:00:32.787ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
22152025-11-13T08:00:32.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
22162025-11-13T08:00:32.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
22172025-11-13T08:00:32.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
22182025-11-13T08:00:32.789ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
22192025-11-13T08:00:32.789ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
22202025-11-13T08:00:32.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
22212025-11-13T08:00:32.792ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
22222025-11-13T08:00:32.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
22232025-11-13T08:00:32.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
22242025-11-13T08:00:32.792ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
22252025-11-13T08:00:32.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
22262025-11-13T08:00:32.793ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
22272025-11-13T08:00:32.793ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
22282025-11-13T08:00:32.794ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b517732-16a6-4b2e-997c-ae9731253ba6", "content-length": "26", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22292025-11-13T08:00:32.794ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
22302025-11-13T08:00:32.796ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
22312025-11-13T08:00:32.800ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 158.128299ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
22322025-11-13T08:00:32.804ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
22332025-11-13T08:00:32.804ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
22342025-11-13T08:00:32.804ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
22352025-11-13T08:00:32.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
22362025-11-13T08:00:32.820ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
22372025-11-13T08:00:32.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
22382025-11-13T08:00:32.821ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
22392025-11-13T08:00:32.821ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22402025-11-13T08:00:32.821ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22412025-11-13T08:00:32.821ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "321fa540-312e-42c3-b9f8-ded9f38871f1", "content-length": "372", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22422025-11-13T08:00:32.821ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22432025-11-13T08:00:32.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22442025-11-13T08:00:32.823ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
22452025-11-13T08:00:32.823ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
22462025-11-13T08:00:32.823ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 175.712794ms iteration = 7 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
22472025-11-13T08:00:32.830ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 182.440417ms iteration = 4 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
22482025-11-13T08:00:32.842ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
22492025-11-13T08:00:32.842ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
22502025-11-13T08:00:32.843ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c4c57d8-3a1f-49dd-83d9-4f29f592031e", "content-length": "81", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22512025-11-13T08:00:32.847ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
22522025-11-13T08:00:32.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
22532025-11-13T08:00:32.849ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
22542025-11-13T08:00:32.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
22552025-11-13T08:00:32.856ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
22562025-11-13T08:00:32.856ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
22572025-11-13T08:00:32.857ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c565384-3af7-4bdd-8bce-03878f55cc10", "content-length": "81", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22582025-11-13T08:00:32.861ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
22592025-11-13T08:00:32.861ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
22602025-11-13T08:00:32.861ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 219.335182ms iteration = 4 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
22612025-11-13T08:00:32.862ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 254.938292ms iteration = 4 status = {}
22622025-11-13T08:00:32.868ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6041, tv_nsec: 220507986 } })
22632025-11-13T08:00:32.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22642025-11-13T08:00:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
22652025-11-13T08:00:32.870ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22662025-11-13T08:00:32.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 224.028884ms iteration = 4 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
22672025-11-13T08:00:32.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22682025-11-13T08:00:32.873ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22692025-11-13T08:00:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
22702025-11-13T08:00:32.874ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22712025-11-13T08:00:32.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
22722025-11-13T08:00:32.877ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
22732025-11-13T08:00:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
22742025-11-13T08:00:32.879ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
22752025-11-13T08:00:32.879ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
22762025-11-13T08:00:32.879ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "244fa813-ebeb-4606-a2f6-480e588e023b", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22772025-11-13T08:00:32.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22782025-11-13T08:00:32.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22792025-11-13T08:00:32.881ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22802025-11-13T08:00:32.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22812025-11-13T08:00:32.887ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
22822025-11-13T08:00:32.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
22832025-11-13T08:00:32.892ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
22842025-11-13T08:00:32.892ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
22852025-11-13T08:00:32.892ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "2270c807-39c3-4d70-a984-33d546aaad06", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22862025-11-13T08:00:32.895ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 253.141547ms iteration = 6 status = {}
22872025-11-13T08:00:32.898ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
22882025-11-13T08:00:32.898ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
22892025-11-13T08:00:32.899ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "52f61965-825a-491e-a80f-16a7ac1084a0", "content-length": "81", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22902025-11-13T08:00:32.901ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 253.446583ms iteration = 4 status = {"disabled":false,"errors":[],"sps":[]}
22912025-11-13T08:00:32.934ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
22922025-11-13T08:00:32.934ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe6268f0-083e-4a6f-a881-61aace4b14ac", "content-length": "691", "date": "Thu, 13 Nov 2025 08:00:32 GMT"} })
22932025-11-13T08:00:32.934ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
22942025-11-13T08:00:32.934ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
22952025-11-13T08:00:32.934ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
22962025-11-13T08:00:32.936ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 294.141443ms iteration = 4 status = {}
22972025-11-13T08:00:32.951ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
22982025-11-13T08:00:32.951ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
22992025-11-13T08:00:32.953ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
23002025-11-13T08:00:32.953ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23012025-11-13T08:00:32.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23022025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
23032025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
23042025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
23052025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
23062025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23072025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23082025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23092025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23102025-11-13T08:00:32.969ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
23112025-11-13T08:00:32.970ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
23122025-11-13T08:00:32.970ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
23132025-11-13T08:00:32.970ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23142025-11-13T08:00:32.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23152025-11-13T08:00:32.972ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
23162025-11-13T08:00:32.972ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
23172025-11-13T08:00:33.227ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bc3cd6c1-0be1-48fd-a6de-dcb48f73c916", "date": "Thu, 13 Nov 2025 08:00:33 GMT"} })
23182025-11-13T08:00:33.227ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
23192025-11-13T08:00:33.227ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
23202025-11-13T08:00:33.227ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3fe4eaa-de54-476c-a907-3d0ee6f4fec2", "content-length": "682", "date": "Thu, 13 Nov 2025 08:00:33 GMT"} })
23212025-11-13T08:00:33.228ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23222025-11-13T08:00:33.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23232025-11-13T08:00:33.230ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23242025-11-13T08:00:33.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23252025-11-13T08:00:33.233ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23262025-11-13T08:00:33.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23272025-11-13T08:00:33.236ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
23282025-11-13T08:00:33.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
23292025-11-13T08:00:33.243ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 601.119828ms iteration = 4 status = {}
23302025-11-13T08:00:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 311 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = a6ed377c-1149-467c-9eb4-efa0cd79dd72 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
23312025-11-13T08:00:37.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 20 reason = Timeout
23322025-11-13T08:00:37.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23332025-11-13T08:00:37.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23342025-11-13T08:00:37.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23352025-11-13T08:00:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
23362025-11-13T08:00:37.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23372025-11-13T08:00:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23382025-11-13T08:00:37.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
23392025-11-13T08:00:37.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.861715ms iteration = 20 status = {"config_updated":false}
23402025-11-13T08:00:42.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 11 reason = Timeout
23412025-11-13T08:00:42.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 21 reason = Timeout
23422025-11-13T08:00:42.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23432025-11-13T08:00:42.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23442025-11-13T08:00:42.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23452025-11-13T08:00:42.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23462025-11-13T08:00:42.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23472025-11-13T08:00:42.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23482025-11-13T08:00:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23492025-11-13T08:00:42.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23502025-11-13T08:00:42.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
23512025-11-13T08:00:42.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23522025-11-13T08:00:42.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23532025-11-13T08:00:42.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23542025-11-13T08:00:42.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
23552025-11-13T08:00:42.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.777323ms iteration = 21 status = {"config_updated":false}
23562025-11-13T08:00:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23572025-11-13T08:00:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23582025-11-13T08:00:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23592025-11-13T08:00:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
23602025-11-13T08:00:42.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23612025-11-13T08:00:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
23622025-11-13T08:00:42.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 11 reason = Timeout
23632025-11-13T08:00:42.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23642025-11-13T08:00:42.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23652025-11-13T08:00:42.644ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
23662025-11-13T08:00:42.644ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.546002ms iteration = 11 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
23672025-11-13T08:00:42.677ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
23682025-11-13T08:00:42.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
23692025-11-13T08:00:42.712ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 70.732861ms iteration = 11 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
23702025-11-13T08:00:45.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 2af9e387-3f5f-4509-8881-d1c7089dbb27 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
23712025-11-13T08:00:45.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 2af9e387-3f5f-4509-8881-d1c7089dbb27 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
23722025-11-13T08:00:45.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 2af9e387-3f5f-4509-8881-d1c7089dbb27 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
23732025-11-13T08:00:45.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3546 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 2af9e387-3f5f-4509-8881-d1c7089dbb27 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
23742025-11-13T08:00:45.477ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = f51fd10e-0b16-4eda-9e6f-5fc85040b5fe uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
23752025-11-13T08:00:45.477ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = f51fd10e-0b16-4eda-9e6f-5fc85040b5fe roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
23762025-11-13T08:00:45.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = f51fd10e-0b16-4eda-9e6f-5fc85040b5fe resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
23772025-11-13T08:00:45.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2642 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = f51fd10e-0b16-4eda-9e6f-5fc85040b5fe response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
23782025-11-13T08:00:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 292 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = f769b054-befb-4601-980c-40a34419fdab response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
23792025-11-13T08:00:47.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 11 reason = Timeout
23802025-11-13T08:00:47.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 22 reason = Timeout
23812025-11-13T08:00:47.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
23822025-11-13T08:00:47.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23832025-11-13T08:00:47.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23842025-11-13T08:00:47.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
23852025-11-13T08:00:47.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23862025-11-13T08:00:47.620ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23872025-11-13T08:00:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
23882025-11-13T08:00:47.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
23892025-11-13T08:00:47.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
23902025-11-13T08:00:47.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
23912025-11-13T08:00:47.624ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
23922025-11-13T08:00:47.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
23932025-11-13T08:00:47.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
23942025-11-13T08:00:47.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.043088ms iteration = 22 status = {"config_updated":false}
23952025-11-13T08:00:47.626ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6ebc3749-17f1-41eb-93de-ab321c41fd59 original_time_started = 2025-11-13 08:00:04.446770 UTC
23962025-11-13T08:00:47.626ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.551979ms iteration = 11 status = {"Loaded":{"collection_id":"6ebc3749-17f1-41eb-93de-ab321c41fd59","time_loaded":"2025-11-13T08:00:47.618043239Z","time_started":"2025-11-13T08:00:04.446770Z"}}
23972025-11-13T08:00:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 8 reason = Timeout
23982025-11-13T08:00:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 8 reason = Timeout
23992025-11-13T08:00:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
24002025-11-13T08:00:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
24012025-11-13T08:00:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
24022025-11-13T08:00:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
24032025-11-13T08:00:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
24042025-11-13T08:00:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
24052025-11-13T08:00:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24062025-11-13T08:00:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24072025-11-13T08:00:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
24082025-11-13T08:00:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
24092025-11-13T08:00:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
24102025-11-13T08:00:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
24112025-11-13T08:00:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.59159ms iteration = 8 status = "NoSitrep"
24122025-11-13T08:00:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
24132025-11-13T08:00:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
24142025-11-13T08:00:47.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
24152025-11-13T08:00:47.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
24162025-11-13T08:00:47.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24172025-11-13T08:00:47.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
24182025-11-13T08:00:47.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
24192025-11-13T08:00:47.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.440974ms iteration = 8 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
24202025-11-13T08:00:52.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 12 reason = Timeout
24212025-11-13T08:00:52.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 23 reason = Timeout
24222025-11-13T08:00:52.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24232025-11-13T08:00:52.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
24242025-11-13T08:00:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24252025-11-13T08:00:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
24262025-11-13T08:00:52.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24272025-11-13T08:00:52.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24282025-11-13T08:00:52.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
24292025-11-13T08:00:52.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24302025-11-13T08:00:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24312025-11-13T08:00:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
24322025-11-13T08:00:52.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
24332025-11-13T08:00:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
24342025-11-13T08:00:52.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
24352025-11-13T08:00:52.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.395665ms iteration = 23 status = {"config_updated":false}
24362025-11-13T08:00:52.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24372025-11-13T08:00:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24382025-11-13T08:00:52.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24392025-11-13T08:00:52.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
24402025-11-13T08:00:52.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
24412025-11-13T08:00:52.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
24422025-11-13T08:00:52.642ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
24432025-11-13T08:00:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 12 reason = Timeout
24442025-11-13T08:00:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24452025-11-13T08:00:52.642ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.229276ms iteration = 12 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
24462025-11-13T08:00:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24472025-11-13T08:00:52.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
24482025-11-13T08:00:52.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
24492025-11-13T08:00:52.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.991378ms iteration = 12 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
24502025-11-13T08:00:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 487 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = e451e06a-8190-4513-9484-82b5085862c8 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
24512025-11-13T08:00:57.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 24 reason = Timeout
24522025-11-13T08:00:57.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
24532025-11-13T08:00:57.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
24542025-11-13T08:00:57.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
24552025-11-13T08:00:57.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
24562025-11-13T08:00:57.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
24572025-11-13T08:00:57.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
24582025-11-13T08:00:57.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
24592025-11-13T08:00:57.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.878175ms iteration = 24 status = {"config_updated":false}
24602025-11-13T08:01:00.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 54c316f6-1e69-47ae-b868-ea26a618ec28 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
24612025-11-13T08:01:00.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 54c316f6-1e69-47ae-b868-ea26a618ec28 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
24622025-11-13T08:01:00.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 54c316f6-1e69-47ae-b868-ea26a618ec28 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
24632025-11-13T08:01:00.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2358 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = 54c316f6-1e69-47ae-b868-ea26a618ec28 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
24642025-11-13T08:01:00.475ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = a533f2d2-4fc7-4191-92af-916edff9a78c uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
24652025-11-13T08:01:00.475ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = a533f2d2-4fc7-4191-92af-916edff9a78c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
24662025-11-13T08:01:00.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = a533f2d2-4fc7-4191-92af-916edff9a78c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
24672025-11-13T08:01:00.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2166 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:44347 req_id = a533f2d2-4fc7-4191-92af-916edff9a78c response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
24682025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 5 reason = Timeout
24692025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 3 reason = Timeout
24702025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
24712025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
24722025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 3 reason = Timeout
24732025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 5 reason = Timeout
24742025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 3 reason = Timeout
24752025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 3 reason = Timeout
24762025-11-13T08:01:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:51:02.607204207 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
24772025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
24782025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 3 reason = Timeout
24792025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
24802025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
24812025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 3 reason = Timeout
24822025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 5 reason = Timeout
24832025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 5 reason = Timeout
24842025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
24852025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
24862025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 5 reason = Timeout
24872025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
24882025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
24892025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
24902025-11-13T08:01:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
24912025-11-13T08:01:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
24922025-11-13T08:01:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
24932025-11-13T08:01:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
24942025-11-13T08:01:02.610ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 4 reason = Timeout
24952025-11-13T08:01:02.610ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
24962025-11-13T08:01:02.610ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
24972025-11-13T08:01:02.610ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = ExternalNetworking step id = Ensure
24982025-11-13T08:01:02.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
24992025-11-13T08:01:02.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
25002025-11-13T08:01:02.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25012025-11-13T08:01:02.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6070, tv_nsec: 963273823 } })
25022025-11-13T08:01:02.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6070, tv_nsec: 963282693 } })
25032025-11-13T08:01:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25042025-11-13T08:01:02.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25052025-11-13T08:01:02.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
25062025-11-13T08:01:02.611ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
25072025-11-13T08:01:02.611ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 4.269694ms iteration = 3 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
25082025-11-13T08:01:02.611ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
25092025-11-13T08:01:02.611ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.377167ms iteration = 3 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
25102025-11-13T08:01:02.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25112025-11-13T08:01:02.614ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6070, tv_nsec: 966396684 } })
25122025-11-13T08:01:02.614ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6070, tv_nsec: 967106249 } })
25132025-11-13T08:01:02.615ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
25142025-11-13T08:01:02.617ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:51:02.607204207 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
25152025-11-13T08:01:02.617ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 9.877724ms iteration = 3 status = {"expiration":"2025-11-13T07:51:02.607204207Z","pruned":[]}
25162025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 3 reason = Timeout
25172025-11-13T08:01:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
25182025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 9 reason = Timeout
25192025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
25202025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 13 reason = Timeout
25212025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
25222025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 12 reason = Timeout
25232025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25242025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
25252025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 9 reason = Timeout
25262025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
25272025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 7 reason = Timeout
25282025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25292025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 25 reason = Timeout
25302025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25312025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
25322025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
25332025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
25342025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
25352025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25362025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25372025-11-13T08:01:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25382025-11-13T08:01:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25392025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25402025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
25412025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
25422025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
25432025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25442025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25452025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25462025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25472025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25482025-11-13T08:01:02.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25492025-11-13T08:01:02.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25502025-11-13T08:01:02.626ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25512025-11-13T08:01:02.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25522025-11-13T08:01:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
25532025-11-13T08:01:02.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
25542025-11-13T08:01:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
25552025-11-13T08:01:02.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25562025-11-13T08:01:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
25572025-11-13T08:01:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25582025-11-13T08:01:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
25592025-11-13T08:01:02.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
25602025-11-13T08:01:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
25612025-11-13T08:01:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
25622025-11-13T08:01:02.630ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
25632025-11-13T08:01:02.630ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
25642025-11-13T08:01:02.630ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25652025-11-13T08:01:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
25662025-11-13T08:01:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
25672025-11-13T08:01:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
25682025-11-13T08:01:02.631ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
25692025-11-13T08:01:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
25702025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
25712025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 27.145454ms iteration = 5 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
25722025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
25732025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 16.155518ms iteration = 25 status = {"config_updated":false}
25742025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
25752025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
25762025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25772025-11-13T08:01:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
25782025-11-13T08:01:02.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
25792025-11-13T08:01:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
25802025-11-13T08:01:02.637ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6ebc3749-17f1-41eb-93de-ab321c41fd59 original_time_started = 2025-11-13 08:00:04.446770 UTC
25812025-11-13T08:01:02.637ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 19.229998ms iteration = 12 status = {"Loaded":{"collection_id":"6ebc3749-17f1-41eb-93de-ab321c41fd59","time_loaded":"2025-11-13T08:01:02.618173933Z","time_started":"2025-11-13T08:00:04.446770Z"}}
25822025-11-13T08:01:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
25832025-11-13T08:01:02.638ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
25842025-11-13T08:01:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
25852025-11-13T08:01:02.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25862025-11-13T08:01:02.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
25872025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 3 reason = Timeout
25882025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
25892025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 5 reason = Timeout
25902025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
25912025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 5 reason = Timeout
25922025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
25932025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
25942025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 3 reason = Timeout
25952025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
25962025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 5 reason = Timeout
25972025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 5 reason = Timeout
25982025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
25992025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 5 reason = Timeout
26002025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26012025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
26022025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 7 reason = Timeout
26032025-11-13T08:01:02.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
26042025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26052025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 5 reason = Timeout
26062025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26072025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 13 reason = Timeout
26082025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26092025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26102025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
26112025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26122025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
26132025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
26142025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
26152025-11-13T08:01:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
26162025-11-13T08:01:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26172025-11-13T08:01:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
26182025-11-13T08:01:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26192025-11-13T08:01:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 3 reason = Timeout
26202025-11-13T08:01:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
26212025-11-13T08:01:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
26222025-11-13T08:01:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
26232025-11-13T08:01:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
26242025-11-13T08:01:02.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
26252025-11-13T08:01:02.645ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26262025-11-13T08:01:02.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26272025-11-13T08:01:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
26282025-11-13T08:01:02.646ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26292025-11-13T08:01:02.647ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26302025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
26312025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26322025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
26332025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 5 reason = Timeout
26342025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 3 reason = Timeout
26352025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26362025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
26372025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 5 reason = Timeout
26382025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 9 reason = Timeout
26392025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 5 reason = Timeout
26402025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
26412025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
26422025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 9 reason = Timeout
26432025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 5 reason = Timeout
26442025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
26452025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26462025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 3 reason = Timeout
26472025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
26482025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 5 reason = Timeout
26492025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
26502025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 5 reason = Timeout
26512025-11-13T08:01:02.647ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
26522025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
26532025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26542025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
26552025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
26562025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
26572025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26582025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
26592025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
26602025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
26612025-11-13T08:01:02.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6071, tv_nsec: 1017715 } })
26622025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 3 reason = Timeout
26632025-11-13T08:01:02.648ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
26642025-11-13T08:01:02.648ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
26652025-11-13T08:01:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
26662025-11-13T08:01:02.650ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
26672025-11-13T08:01:02.650ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
26682025-11-13T08:01:02.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
26692025-11-13T08:01:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
26702025-11-13T08:01:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
26712025-11-13T08:01:02.651ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
26722025-11-13T08:01:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
26732025-11-13T08:01:02.651ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
26742025-11-13T08:01:02.652ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
26752025-11-13T08:01:02.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
26762025-11-13T08:01:02.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
26772025-11-13T08:01:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
26782025-11-13T08:01:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
26792025-11-13T08:01:02.655ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
26802025-11-13T08:01:02.656ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
26812025-11-13T08:01:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
26822025-11-13T08:01:02.656ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 11.335033ms iteration = 3 status = {"errors":[],"found_port_ok":[]}
26832025-11-13T08:01:02.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26842025-11-13T08:01:02.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
26852025-11-13T08:01:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
26862025-11-13T08:01:02.657ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
26872025-11-13T08:01:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
26882025-11-13T08:01:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
26892025-11-13T08:01:02.660ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
26902025-11-13T08:01:02.660ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
26912025-11-13T08:01:02.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
26922025-11-13T08:01:02.661ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
26932025-11-13T08:01:02.661ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 13.251761ms iteration = 3 status = {"by_rx":{},"error":null}
26942025-11-13T08:01:02.664ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 16.903082ms iteration = 5 status = {"errors":[],"finish_invoked_ok":[]}
26952025-11-13T08:01:02.667ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
26962025-11-13T08:01:02.668ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
26972025-11-13T08:01:02.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
26982025-11-13T08:01:02.670ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
26992025-11-13T08:01:02.675ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
27002025-11-13T08:01:02.678ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
27012025-11-13T08:01:02.689ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
27022025-11-13T08:01:02.690ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
27032025-11-13T08:01:02.691ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
27042025-11-13T08:01:02.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
27052025-11-13T08:01:02.694ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27062025-11-13T08:01:02.697ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
27072025-11-13T08:01:02.697ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
27082025-11-13T08:01:02.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
27092025-11-13T08:01:02.699ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
27102025-11-13T08:01:02.708ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27112025-11-13T08:01:02.713ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
27122025-11-13T08:01:02.715ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27132025-11-13T08:01:02.716ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
27142025-11-13T08:01:02.716ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
27152025-11-13T08:01:02.716ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
27162025-11-13T08:01:02.716ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27172025-11-13T08:01:02.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27182025-11-13T08:01:02.721ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27192025-11-13T08:01:02.724ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27202025-11-13T08:01:02.727ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
27212025-11-13T08:01:02.728ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
27222025-11-13T08:01:02.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
27232025-11-13T08:01:02.728ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
27242025-11-13T08:01:02.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
27252025-11-13T08:01:02.729ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
27262025-11-13T08:01:02.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
27272025-11-13T08:01:02.730ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
27282025-11-13T08:01:02.731ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = SledList step id = Fetch
27292025-11-13T08:01:02.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27302025-11-13T08:01:02.731ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
27312025-11-13T08:01:02.731ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 113.112364ms iteration = 13 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
27322025-11-13T08:01:02.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27332025-11-13T08:01:02.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27342025-11-13T08:01:02.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
27352025-11-13T08:01:02.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27362025-11-13T08:01:02.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27372025-11-13T08:01:02.740ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6071, tv_nsec: 92609946 } })
27382025-11-13T08:01:02.740ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
27392025-11-13T08:01:02.744ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27402025-11-13T08:01:02.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
27412025-11-13T08:01:02.745ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27422025-11-13T08:01:02.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27432025-11-13T08:01:02.747ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27442025-11-13T08:01:02.747ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 99.904324ms iteration = 9 status = "NoSitrep"
27452025-11-13T08:01:02.749ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27462025-11-13T08:01:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27472025-11-13T08:01:02.750ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27482025-11-13T08:01:02.750ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
27492025-11-13T08:01:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
27502025-11-13T08:01:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
27512025-11-13T08:01:02.752ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27522025-11-13T08:01:02.753ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27532025-11-13T08:01:02.754ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
27542025-11-13T08:01:02.754ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 111.773558ms iteration = 5 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
27552025-11-13T08:01:02.754ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
27562025-11-13T08:01:02.754ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
27572025-11-13T08:01:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
27582025-11-13T08:01:02.755ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 107.672407ms iteration = 3 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
27592025-11-13T08:01:02.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27602025-11-13T08:01:02.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27612025-11-13T08:01:02.756ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
27622025-11-13T08:01:02.756ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
27632025-11-13T08:01:02.756ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 113.980391ms iteration = 3 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
27642025-11-13T08:01:02.756ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27652025-11-13T08:01:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
27662025-11-13T08:01:02.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27672025-11-13T08:01:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27682025-11-13T08:01:02.760ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 112.151775ms iteration = 5 status = {"errors":[],"garbage_collect_requested":[]}
27692025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
27702025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
27712025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
27722025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
27732025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
27742025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
27752025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
27762025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
27772025-11-13T08:01:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
27782025-11-13T08:01:02.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
27792025-11-13T08:01:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
27802025-11-13T08:01:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
27812025-11-13T08:01:02.763ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
27822025-11-13T08:01:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
27832025-11-13T08:01:02.766ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 124.140231ms iteration = 5 status = 0
27842025-11-13T08:01:02.766ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27852025-11-13T08:01:02.767ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
27862025-11-13T08:01:02.767ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
27872025-11-13T08:01:02.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
27882025-11-13T08:01:02.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
27892025-11-13T08:01:02.768ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
27902025-11-13T08:01:02.768ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
27912025-11-13T08:01:02.768ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
27922025-11-13T08:01:02.768ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
27932025-11-13T08:01:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
27942025-11-13T08:01:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
27952025-11-13T08:01:02.769ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
27962025-11-13T08:01:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
27972025-11-13T08:01:02.770ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
27982025-11-13T08:01:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
27992025-11-13T08:01:02.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
28002025-11-13T08:01:02.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
28012025-11-13T08:01:02.772ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28022025-11-13T08:01:02.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28032025-11-13T08:01:02.773ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28042025-11-13T08:01:02.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28052025-11-13T08:01:02.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28062025-11-13T08:01:02.776ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
28072025-11-13T08:01:02.777ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
28082025-11-13T08:01:02.777ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
28092025-11-13T08:01:02.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
28102025-11-13T08:01:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
28112025-11-13T08:01:02.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
28122025-11-13T08:01:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
28132025-11-13T08:01:02.780ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
28142025-11-13T08:01:02.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28152025-11-13T08:01:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28162025-11-13T08:01:02.781ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
28172025-11-13T08:01:02.781ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 132.509916ms iteration = 9 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
28182025-11-13T08:01:02.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28192025-11-13T08:01:02.786ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
28202025-11-13T08:01:02.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
28212025-11-13T08:01:02.786ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
28222025-11-13T08:01:02.786ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
28232025-11-13T08:01:02.786ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
28242025-11-13T08:01:02.786ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 179.674003ms iteration = 3 status = {"generation":1}
28252025-11-13T08:01:02.787ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
28262025-11-13T08:01:02.787ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
28272025-11-13T08:01:02.787ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
28282025-11-13T08:01:02.787ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 180.775584ms iteration = 3 status = {"generation":2}
28292025-11-13T08:01:02.788ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
28302025-11-13T08:01:02.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28312025-11-13T08:01:02.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28322025-11-13T08:01:02.793ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
28332025-11-13T08:01:02.793ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
28342025-11-13T08:01:02.793ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 175.45248ms iteration = 3 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
28352025-11-13T08:01:02.801ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
28362025-11-13T08:01:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
28372025-11-13T08:01:02.810ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 162.419374ms iteration = 5 status = {"errors":[],"requests_created_ok":[]}
28382025-11-13T08:01:02.813ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
28392025-11-13T08:01:02.813ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
28402025-11-13T08:01:02.813ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 171.056683ms iteration = 5 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
28412025-11-13T08:01:02.837ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
28422025-11-13T08:01:02.837ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfbfa2f3-8a9a-4b00-a25b-db89b9b9ccd1", "content-length": "78", "date": "Thu, 13 Nov 2025 08:01:02 GMT"} })
28432025-11-13T08:01:02.838ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
28442025-11-13T08:01:02.838ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
28452025-11-13T08:01:02.838ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
28462025-11-13T08:01:02.838ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 195.81237ms iteration = 13 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
28472025-11-13T08:01:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
28482025-11-13T08:01:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
28492025-11-13T08:01:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
28502025-11-13T08:01:02.843ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
28512025-11-13T08:01:02.843ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
28522025-11-13T08:01:02.843ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
28532025-11-13T08:01:02.847ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
28542025-11-13T08:01:02.847ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
28552025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
28562025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
28572025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28582025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
28592025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
28602025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
28612025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
28622025-11-13T08:01:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
28632025-11-13T08:01:02.851ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
28642025-11-13T08:01:02.851ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
28652025-11-13T08:01:02.851ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "b47f6798-fac7-427e-b8b5-bb1010b181c3", "content-length": "81", "date": "Thu, 13 Nov 2025 08:01:02 GMT"} })
28662025-11-13T08:01:02.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
28672025-11-13T08:01:02.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28682025-11-13T08:01:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28692025-11-13T08:01:02.853ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 205.354617ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
28702025-11-13T08:01:02.858ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = DeployNexusRecords step id = Ensure
28712025-11-13T08:01:02.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28722025-11-13T08:01:02.858ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
28732025-11-13T08:01:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28742025-11-13T08:01:02.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28752025-11-13T08:01:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
28762025-11-13T08:01:02.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28772025-11-13T08:01:02.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28782025-11-13T08:01:02.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
28792025-11-13T08:01:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
28802025-11-13T08:01:02.871ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28812025-11-13T08:01:02.871ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
28822025-11-13T08:01:02.871ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 252.931383ms iteration = 7 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
28832025-11-13T08:01:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
28842025-11-13T08:01:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28852025-11-13T08:01:02.874ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28862025-11-13T08:01:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
28872025-11-13T08:01:02.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
28882025-11-13T08:01:02.890ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
28892025-11-13T08:01:02.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
28902025-11-13T08:01:02.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28912025-11-13T08:01:02.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28922025-11-13T08:01:02.893ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 250.814511ms iteration = 5 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
28932025-11-13T08:01:02.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
28942025-11-13T08:01:02.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
28952025-11-13T08:01:02.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
28962025-11-13T08:01:02.907ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
28972025-11-13T08:01:02.907ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
28982025-11-13T08:01:02.908ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b6c47b7-a782-409f-beaf-a2257b39dead", "content-length": "26", "date": "Thu, 13 Nov 2025 08:01:02 GMT"} })
28992025-11-13T08:01:02.908ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29002025-11-13T08:01:02.910ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
29012025-11-13T08:01:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29022025-11-13T08:01:02.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29032025-11-13T08:01:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29042025-11-13T08:01:02.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29052025-11-13T08:01:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29062025-11-13T08:01:02.917ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
29072025-11-13T08:01:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
29082025-11-13T08:01:02.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29092025-11-13T08:01:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
29102025-11-13T08:01:02.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29112025-11-13T08:01:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29122025-11-13T08:01:02.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29132025-11-13T08:01:02.941ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29142025-11-13T08:01:02.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29152025-11-13T08:01:02.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29162025-11-13T08:01:02.942ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
29172025-11-13T08:01:02.942ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
29182025-11-13T08:01:02.943ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f4deb21-a5d6-4d38-80b2-1f6f2a5a666f", "content-length": "81", "date": "Thu, 13 Nov 2025 08:01:02 GMT"} })
29192025-11-13T08:01:02.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29202025-11-13T08:01:02.949ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
29212025-11-13T08:01:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29222025-11-13T08:01:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
29232025-11-13T08:01:02.951ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
29242025-11-13T08:01:02.951ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
29252025-11-13T08:01:02.952ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c54396a-1c0b-46d4-ab18-f9ee006432d7", "content-length": "81", "date": "Thu, 13 Nov 2025 08:01:02 GMT"} })
29262025-11-13T08:01:02.953ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29272025-11-13T08:01:02.956ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 308.602467ms iteration = 5 status = {"disabled":false,"errors":[],"sps":[]}
29282025-11-13T08:01:02.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29292025-11-13T08:01:02.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
29302025-11-13T08:01:02.957ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
29312025-11-13T08:01:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
29322025-11-13T08:01:02.959ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
29332025-11-13T08:01:02.960ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
29342025-11-13T08:01:02.962ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
29352025-11-13T08:01:02.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29362025-11-13T08:01:02.962ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
29372025-11-13T08:01:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29382025-11-13T08:01:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
29392025-11-13T08:01:02.963ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 356.144522ms iteration = 5 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
29402025-11-13T08:01:02.963ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = SledAgent step id = Ensure
29412025-11-13T08:01:02.964ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
29422025-11-13T08:01:02.964ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
29432025-11-13T08:01:02.964ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d60137c-69d0-473a-984d-74ed05d7324f", "content-length": "26", "date": "Thu, 13 Nov 2025 08:01:02 GMT"} })
29442025-11-13T08:01:02.964ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
29452025-11-13T08:01:02.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29462025-11-13T08:01:02.966ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
29472025-11-13T08:01:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29482025-11-13T08:01:02.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29492025-11-13T08:01:02.971ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
29502025-11-13T08:01:02.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29512025-11-13T08:01:02.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29522025-11-13T08:01:02.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29532025-11-13T08:01:02.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29542025-11-13T08:01:02.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29552025-11-13T08:01:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
29562025-11-13T08:01:02.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29572025-11-13T08:01:02.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29582025-11-13T08:01:02.983ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
29592025-11-13T08:01:02.983ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 376.424321ms iteration = 3 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
29602025-11-13T08:01:03.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29612025-11-13T08:01:03.011ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 363.127829ms iteration = 5 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
29622025-11-13T08:01:03.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29632025-11-13T08:01:03.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29642025-11-13T08:01:03.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29652025-11-13T08:01:03.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29662025-11-13T08:01:03.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29672025-11-13T08:01:03.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29682025-11-13T08:01:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29692025-11-13T08:01:03.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29702025-11-13T08:01:03.028ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29712025-11-13T08:01:03.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29722025-11-13T08:01:03.029ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29732025-11-13T08:01:03.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29742025-11-13T08:01:03.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29752025-11-13T08:01:03.031ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29762025-11-13T08:01:03.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29772025-11-13T08:01:03.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29782025-11-13T08:01:03.035ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29792025-11-13T08:01:03.035ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29802025-11-13T08:01:03.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29812025-11-13T08:01:03.036ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 418.751702ms iteration = 9 status = {"physical_disks_added":0}
29822025-11-13T08:01:03.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29832025-11-13T08:01:03.038ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29842025-11-13T08:01:03.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29852025-11-13T08:01:03.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29862025-11-13T08:01:03.040ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
29872025-11-13T08:01:03.040ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
29882025-11-13T08:01:03.041ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
29892025-11-13T08:01:03.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29902025-11-13T08:01:03.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29912025-11-13T08:01:03.047ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29922025-11-13T08:01:03.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29932025-11-13T08:01:03.054ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
29942025-11-13T08:01:03.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29952025-11-13T08:01:03.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29962025-11-13T08:01:03.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29972025-11-13T08:01:03.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
29982025-11-13T08:01:03.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
29992025-11-13T08:01:03.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30002025-11-13T08:01:03.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30012025-11-13T08:01:03.069ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
30022025-11-13T08:01:03.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30032025-11-13T08:01:03.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30042025-11-13T08:01:03.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30052025-11-13T08:01:03.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30062025-11-13T08:01:03.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30072025-11-13T08:01:03.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
30082025-11-13T08:01:03.158ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
30092025-11-13T08:01:03.158ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 515.928563ms iteration = 3 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
30102025-11-13T08:01:03.184ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
30112025-11-13T08:01:03.185ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "98314cf1-88b3-428d-b724-e11a39495b87", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30122025-11-13T08:01:03.185ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30132025-11-13T08:01:03.186ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "3f0121c7-75f5-4920-a69a-38efd6a49ddf", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30142025-11-13T08:01:03.186ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30152025-11-13T08:01:03.188ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
30162025-11-13T08:01:03.188ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 581.842649ms iteration = 5 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
30172025-11-13T08:01:03.201ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
30182025-11-13T08:01:03.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30192025-11-13T08:01:03.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30202025-11-13T08:01:03.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30212025-11-13T08:01:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30222025-11-13T08:01:03.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30232025-11-13T08:01:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30242025-11-13T08:01:03.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30252025-11-13T08:01:03.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30262025-11-13T08:01:03.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30272025-11-13T08:01:03.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30282025-11-13T08:01:03.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30292025-11-13T08:01:03.226ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
30302025-11-13T08:01:03.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30312025-11-13T08:01:03.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30322025-11-13T08:01:03.229ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30332025-11-13T08:01:03.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30342025-11-13T08:01:03.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30352025-11-13T08:01:03.235ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30362025-11-13T08:01:03.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30372025-11-13T08:01:03.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30382025-11-13T08:01:03.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30392025-11-13T08:01:03.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30402025-11-13T08:01:03.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30412025-11-13T08:01:03.244ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
30422025-11-13T08:01:03.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30432025-11-13T08:01:03.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30442025-11-13T08:01:03.257ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 649.850576ms iteration = 5 status = {}
30452025-11-13T08:01:03.275ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6071, tv_nsec: 627540213 } })
30462025-11-13T08:01:03.293ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
30472025-11-13T08:01:03.293ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
30482025-11-13T08:01:03.294ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9060d03-b221-4eec-a853-1f0d1bd0c69c", "content-length": "372", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
30492025-11-13T08:01:03.294ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
30502025-11-13T08:01:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
30512025-11-13T08:01:03.295ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
30522025-11-13T08:01:03.297ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "555b0331-7302-4833-9c4c-f5e26db5ab24", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
30532025-11-13T08:01:03.300ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = FirewallRules step id = Ensure
30542025-11-13T08:01:03.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30552025-11-13T08:01:03.301ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
30562025-11-13T08:01:03.301ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
30572025-11-13T08:01:03.301ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 82990133-0319-416d-9a0f-abac7f2c79e8 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
30582025-11-13T08:01:03.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30592025-11-13T08:01:03.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30602025-11-13T08:01:03.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30612025-11-13T08:01:03.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30622025-11-13T08:01:03.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30632025-11-13T08:01:03.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30642025-11-13T08:01:03.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30652025-11-13T08:01:03.306ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30662025-11-13T08:01:03.307ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
30672025-11-13T08:01:03.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
30682025-11-13T08:01:03.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
30692025-11-13T08:01:03.308ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
30702025-11-13T08:01:03.309ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30712025-11-13T08:01:03.309ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30722025-11-13T08:01:03.310ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "71d5142e-31bd-4bc1-b9ae-0fa5ca1d2194", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
30732025-11-13T08:01:03.310ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
30742025-11-13T08:01:03.311ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
30752025-11-13T08:01:03.311ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
30762025-11-13T08:01:03.311ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 692.973504ms iteration = 9 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
30772025-11-13T08:01:03.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30782025-11-13T08:01:03.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30792025-11-13T08:01:03.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30802025-11-13T08:01:03.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30812025-11-13T08:01:03.313ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
30822025-11-13T08:01:03.313ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 665.063285ms iteration = 3 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
30832025-11-13T08:01:03.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30842025-11-13T08:01:03.319ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
30852025-11-13T08:01:03.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30862025-11-13T08:01:03.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30872025-11-13T08:01:03.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30882025-11-13T08:01:03.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30892025-11-13T08:01:03.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30902025-11-13T08:01:03.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
30912025-11-13T08:01:03.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30922025-11-13T08:01:03.332ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
30932025-11-13T08:01:03.332ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a06876aa-abda-4b81-b930-f5c99990518e", "content-length": "691", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
30942025-11-13T08:01:03.332ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
30952025-11-13T08:01:03.332ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
30962025-11-13T08:01:03.332ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
30972025-11-13T08:01:03.333ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
30982025-11-13T08:01:03.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
30992025-11-13T08:01:03.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31002025-11-13T08:01:03.335ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 692.773519ms iteration = 5 status = {}
31012025-11-13T08:01:03.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31022025-11-13T08:01:03.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31032025-11-13T08:01:03.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31042025-11-13T08:01:03.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31052025-11-13T08:01:03.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31062025-11-13T08:01:03.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31072025-11-13T08:01:03.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31082025-11-13T08:01:03.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31092025-11-13T08:01:03.346ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31102025-11-13T08:01:03.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31112025-11-13T08:01:03.348ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
31122025-11-13T08:01:03.348ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31132025-11-13T08:01:03.349ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "cb94c31e-b900-46ef-97dd-150161261677", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
31142025-11-13T08:01:03.349ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
31152025-11-13T08:01:03.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31162025-11-13T08:01:03.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31172025-11-13T08:01:03.351ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "f833c9a8-e53e-4809-b2b6-e4ee9dce8827", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
31182025-11-13T08:01:03.351ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
31192025-11-13T08:01:03.353ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "edc6591e-0fb2-4c2d-b545-ffb21f5e0ae3", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
31202025-11-13T08:01:03.353ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
31212025-11-13T08:01:03.356ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
31222025-11-13T08:01:03.356ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31232025-11-13T08:01:03.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31242025-11-13T08:01:03.358ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
31252025-11-13T08:01:03.358ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
31262025-11-13T08:01:03.358ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "51ce5fc9-537e-44ed-a155-ca3ed323b9fb", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
31272025-11-13T08:01:03.358ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31282025-11-13T08:01:03.358ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
31292025-11-13T08:01:03.358ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 751.543825ms iteration = 5 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
31302025-11-13T08:01:03.358ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31312025-11-13T08:01:03.360ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31322025-11-13T08:01:03.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31332025-11-13T08:01:03.362ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31342025-11-13T08:01:03.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31352025-11-13T08:01:03.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31362025-11-13T08:01:03.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31372025-11-13T08:01:03.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31382025-11-13T08:01:03.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31392025-11-13T08:01:03.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31402025-11-13T08:01:03.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31412025-11-13T08:01:03.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31422025-11-13T08:01:03.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31432025-11-13T08:01:03.369ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
31442025-11-13T08:01:03.369ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
31452025-11-13T08:01:03.370ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31462025-11-13T08:01:03.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31472025-11-13T08:01:03.372ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d480df93-993f-48ce-9e3f-c2ac63cc0410", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
31482025-11-13T08:01:03.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31492025-11-13T08:01:03.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31502025-11-13T08:01:03.374ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 732.48582ms iteration = 7 status = {}
31512025-11-13T08:01:03.374ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 8 reason = Dependency
31522025-11-13T08:01:03.374ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
31532025-11-13T08:01:03.374ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31542025-11-13T08:01:03.374ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31552025-11-13T08:01:03.375ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31562025-11-13T08:01:03.376ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31572025-11-13T08:01:03.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31582025-11-13T08:01:03.379ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31592025-11-13T08:01:03.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31602025-11-13T08:01:03.380ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
31612025-11-13T08:01:03.380ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31622025-11-13T08:01:03.380ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31632025-11-13T08:01:03.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31642025-11-13T08:01:03.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31652025-11-13T08:01:03.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31662025-11-13T08:01:03.383ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
31672025-11-13T08:01:03.383ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
31682025-11-13T08:01:03.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
31692025-11-13T08:01:03.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31702025-11-13T08:01:03.385ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31712025-11-13T08:01:03.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31722025-11-13T08:01:03.392ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
31732025-11-13T08:01:03.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31742025-11-13T08:01:03.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31752025-11-13T08:01:03.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31762025-11-13T08:01:03.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31772025-11-13T08:01:03.400ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31782025-11-13T08:01:03.401ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31792025-11-13T08:01:03.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31802025-11-13T08:01:03.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31812025-11-13T08:01:03.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31822025-11-13T08:01:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31832025-11-13T08:01:03.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31842025-11-13T08:01:03.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31852025-11-13T08:01:03.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31862025-11-13T08:01:03.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31872025-11-13T08:01:03.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31882025-11-13T08:01:03.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31892025-11-13T08:01:03.411ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31902025-11-13T08:01:03.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31912025-11-13T08:01:03.412ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31922025-11-13T08:01:03.414ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
31932025-11-13T08:01:03.414ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
31942025-11-13T08:01:03.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
31952025-11-13T08:01:03.418ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
31962025-11-13T08:01:03.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31972025-11-13T08:01:03.418ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
31982025-11-13T08:01:03.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
31992025-11-13T08:01:03.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32002025-11-13T08:01:03.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32012025-11-13T08:01:03.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32022025-11-13T08:01:03.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32032025-11-13T08:01:03.431ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
32042025-11-13T08:01:03.431ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
32052025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
32062025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
32072025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32082025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32092025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32102025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
32112025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
32122025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
32132025-11-13T08:01:03.432ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
32142025-11-13T08:01:03.432ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
32152025-11-13T08:01:03.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
32162025-11-13T08:01:03.433ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
32172025-11-13T08:01:03.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32182025-11-13T08:01:03.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32192025-11-13T08:01:03.440ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
32202025-11-13T08:01:03.440ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
32212025-11-13T08:01:03.442ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
32222025-11-13T08:01:03.442ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
32232025-11-13T08:01:03.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32242025-11-13T08:01:03.442ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32252025-11-13T08:01:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32262025-11-13T08:01:03.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32272025-11-13T08:01:03.447ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
32282025-11-13T08:01:03.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32292025-11-13T08:01:03.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32302025-11-13T08:01:03.449ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
32312025-11-13T08:01:03.449ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
32322025-11-13T08:01:03.449ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
32332025-11-13T08:01:03.451ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c1838a1-9239-4583-b4b7-eed4d50ef41c", "content-length": "372", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
32342025-11-13T08:01:03.451ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32352025-11-13T08:01:03.451ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32362025-11-13T08:01:03.518ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
32372025-11-13T08:01:03.518ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
32382025-11-13T08:01:03.518ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "bcee56cf-340f-4eab-a083-003ed2c646db", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
32392025-11-13T08:01:03.518ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32402025-11-13T08:01:03.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32412025-11-13T08:01:03.522ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32422025-11-13T08:01:03.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32432025-11-13T08:01:03.525ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
32442025-11-13T08:01:03.525ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
32452025-11-13T08:01:03.528ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
32462025-11-13T08:01:03.528ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
32472025-11-13T08:01:03.534ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
32482025-11-13T08:01:03.534ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
32492025-11-13T08:01:03.535ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "67f726cc-a61b-49e9-aeea-714134d5e436", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
32502025-11-13T08:01:03.536ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 162.068307ms iteration = 8 status = {}
32512025-11-13T08:01:03.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32522025-11-13T08:01:03.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32532025-11-13T08:01:03.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32542025-11-13T08:01:03.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32552025-11-13T08:01:03.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32562025-11-13T08:01:03.577ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
32572025-11-13T08:01:03.578ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
32582025-11-13T08:01:03.579ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "8cf0c5df-6353-473b-9447-882dfc8682c4", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
32592025-11-13T08:01:03.581ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
32602025-11-13T08:01:03.581ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = Dns step id = Ensure
32612025-11-13T08:01:03.581ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
32622025-11-13T08:01:03.581ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32632025-11-13T08:01:03.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32642025-11-13T08:01:03.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32652025-11-13T08:01:03.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32662025-11-13T08:01:03.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32672025-11-13T08:01:03.588ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
32682025-11-13T08:01:03.642ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
32692025-11-13T08:01:03.642ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
32702025-11-13T08:01:03.642ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
32712025-11-13T08:01:03.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32722025-11-13T08:01:03.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32732025-11-13T08:01:03.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32742025-11-13T08:01:03.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
32752025-11-13T08:01:03.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32762025-11-13T08:01:03.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32772025-11-13T08:01:03.652ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
32782025-11-13T08:01:03.652ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
32792025-11-13T08:01:03.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32802025-11-13T08:01:03.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32812025-11-13T08:01:03.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32822025-11-13T08:01:03.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
32832025-11-13T08:01:03.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32842025-11-13T08:01:03.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32852025-11-13T08:01:03.661ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
32862025-11-13T08:01:03.661ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
32872025-11-13T08:01:03.661ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
32882025-11-13T08:01:03.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32892025-11-13T08:01:03.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32902025-11-13T08:01:03.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32912025-11-13T08:01:03.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32922025-11-13T08:01:03.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32932025-11-13T08:01:03.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
32942025-11-13T08:01:03.725ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
32952025-11-13T08:01:03.725ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
32962025-11-13T08:01:03.725ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
32972025-11-13T08:01:03.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
32982025-11-13T08:01:03.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
32992025-11-13T08:01:03.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33002025-11-13T08:01:03.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
33012025-11-13T08:01:03.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33022025-11-13T08:01:03.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33032025-11-13T08:01:03.735ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
33042025-11-13T08:01:03.735ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
33052025-11-13T08:01:03.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33062025-11-13T08:01:03.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33072025-11-13T08:01:03.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33082025-11-13T08:01:03.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
33092025-11-13T08:01:03.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33102025-11-13T08:01:03.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33112025-11-13T08:01:03.744ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "5a2aa016-004b-4716-8da2-f95b944f69b3", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
33122025-11-13T08:01:03.744ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
33132025-11-13T08:01:03.744ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
33142025-11-13T08:01:03.744ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f0a766b-3c44-447c-bab9-afcc4b7bd6c0", "content-length": "682", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
33152025-11-13T08:01:03.744ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33162025-11-13T08:01:03.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33172025-11-13T08:01:03.746ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
33182025-11-13T08:01:03.746ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
33192025-11-13T08:01:03.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33202025-11-13T08:01:03.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33212025-11-13T08:01:03.748ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33222025-11-13T08:01:03.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33232025-11-13T08:01:03.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33242025-11-13T08:01:03.752ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
33252025-11-13T08:01:03.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33262025-11-13T08:01:03.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33272025-11-13T08:01:03.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33282025-11-13T08:01:03.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33292025-11-13T08:01:03.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33302025-11-13T08:01:03.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33312025-11-13T08:01:03.755ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
33322025-11-13T08:01:03.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33332025-11-13T08:01:03.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
33342025-11-13T08:01:03.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33352025-11-13T08:01:03.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33362025-11-13T08:01:03.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
33372025-11-13T08:01:03.763ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.121317454s iteration = 5 status = {}
33382025-11-13T08:01:03.798ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
33392025-11-13T08:01:03.798ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
33402025-11-13T08:01:03.798ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = OmicronZones step id = Cleanup
33412025-11-13T08:01:03.798ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = OmicronZones step id = Cleanup
33422025-11-13T08:01:03.798ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = PhysicalDisks step id = Cleanup
33432025-11-13T08:01:03.798ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = Clickhouse step id = Ensure
33442025-11-13T08:01:03.798ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = Clickhouse step id = Ensure
33452025-11-13T08:01:03.829ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
33462025-11-13T08:01:03.833ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "86e8ddea-9a85-4c13-8663-91a8d0e348be", "date": "Thu, 13 Nov 2025 08:01:03 GMT"} })
33472025-11-13T08:01:03.835ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = SupportBundles step id = Cleanup
33482025-11-13T08:01:03.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33492025-11-13T08:01:03.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33502025-11-13T08:01:03.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33512025-11-13T08:01:03.838ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
33522025-11-13T08:01:03.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33532025-11-13T08:01:03.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33542025-11-13T08:01:03.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33552025-11-13T08:01:03.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33562025-11-13T08:01:03.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33572025-11-13T08:01:03.845ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
33582025-11-13T08:01:03.887ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = OmicronZones step id = Cleanup
33592025-11-13T08:01:03.888ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
33602025-11-13T08:01:03.888ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
33612025-11-13T08:01:03.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33622025-11-13T08:01:03.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33632025-11-13T08:01:03.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33642025-11-13T08:01:03.891ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
33652025-11-13T08:01:03.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33662025-11-13T08:01:03.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33672025-11-13T08:01:03.929ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
33682025-11-13T08:01:03.929ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
33692025-11-13T08:01:03.929ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = Cockroach step id = Ensure
33702025-11-13T08:01:03.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
33712025-11-13T08:01:03.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
33722025-11-13T08:01:03.949ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
33732025-11-13T08:01:03.949ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 6a103b23-5f6c-4f61-ac12-08cb5bdf1238 step component = MgsUpdates step id = Ensure
33742025-11-13T08:01:03.949ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 8 reason = Dependency
33752025-11-13T08:01:03.949ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Dependency
33762025-11-13T08:01:03.949ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33772025-11-13T08:01:03.949ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33782025-11-13T08:01:03.949ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.339124066s iteration = 4 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4370,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":120849275,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":120849215,"secs":0}},"event_index":1,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":120864876,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":126899484,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":126899435,"secs":0}},"event_index":2,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":247782711,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":105506324,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":105506264,"secs":0}},"event_index":3,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":353313106,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":337002636,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":337002586,"secs":0}},"event_index":4,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":690331532,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":280572666,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":280572615,"secs":0}},"event_index":5,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":970922898,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":217179290,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":217179190,"secs":0}},"event_index":6,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188117457,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5000,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4960,"secs":0}},"event_index":7,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188134338,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3090,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3050,"secs":0}},"event_index":8,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188140428,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3740,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3700,"secs":0}},"event_index":9,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188146558,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2320,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2270,"secs":0}},"event_index":10,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":188150788,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36897815,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36897786,"secs":0}},"event_index":11,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":225049983,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":52577583,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":52577543,"secs":0}},"event_index":12,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":277640948,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41504566,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":41504536,"secs":0}},"event_index":13,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":319163774,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19531154,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":19531114,"secs":0}},"event_index":14,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":338707778,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5640,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5610,"secs":0}},"event_index":15,"execution_id":"a738b7ad-f05f-4dff-be09-d98ba8baa2cc","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":338721838,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
33792025-11-13T08:01:03.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
33802025-11-13T08:01:03.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33812025-11-13T08:01:03.951ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33822025-11-13T08:01:03.951ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33832025-11-13T08:01:03.954ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
33842025-11-13T08:01:03.954ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33852025-11-13T08:01:03.954ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
33862025-11-13T08:01:03.954ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33872025-11-13T08:01:03.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33882025-11-13T08:01:03.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
33892025-11-13T08:01:03.989ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33902025-11-13T08:01:03.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33912025-11-13T08:01:03.990ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33922025-11-13T08:01:03.993ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
33932025-11-13T08:01:03.993ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
33942025-11-13T08:01:03.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
33952025-11-13T08:01:03.995ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
33962025-11-13T08:01:03.995ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 46.396322ms iteration = 8 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
33972025-11-13T08:01:04.075ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(badd5a66-f8f5-414c-9527-1e9f4eb27e70, 0), (6fdab7d7-73ae-4e87-b0a9-ced1fd2daeb6, 0), (07399175-748d-4999-b636-5f12bc3d81dd, 0), (6ebc3749-17f1-41eb-93de-ab321c41fd59, 0)] collection_id = badd5a66-f8f5-414c-9527-1e9f4eb27e70
33982025-11-13T08:01:04.075ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33992025-11-13T08:01:04.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
34002025-11-13T08:01:04.284ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = badd5a66-f8f5-414c-9527-1e9f4eb27e70 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
34012025-11-13T08:01:04.285ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34022025-11-13T08:01:04.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
34032025-11-13T08:01:04.289ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(6fdab7d7-73ae-4e87-b0a9-ced1fd2daeb6, 0), (07399175-748d-4999-b636-5f12bc3d81dd, 0), (6ebc3749-17f1-41eb-93de-ab321c41fd59, 0)]
34042025-11-13T08:01:04.289ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6072, tv_nsec: 642169999 } })
34052025-11-13T08:01:04.329ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6072, tv_nsec: 681731307 } })
34062025-11-13T08:01:04.329ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
34072025-11-13T08:01:04.329ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
34082025-11-13T08:01:04.329ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
34092025-11-13T08:01:04.330ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c7743f9-7e51-47c5-a3d9-648694c96a13", "content-length": "476", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34102025-11-13T08:01:04.330ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
34112025-11-13T08:01:04.330ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "275be2bb-3446-4d48-8db0-6e8b674b14ba", "content-length": "734", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34122025-11-13T08:01:04.330ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
34132025-11-13T08:01:04.331ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b717e74f-2d35-400f-8594-aae1bfe85659", "content-length": "111", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34142025-11-13T08:01:04.331ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
34152025-11-13T08:01:04.331ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1564bea3-6bc5-4d69-bf8c-ffd999734c3e", "content-length": "111", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34162025-11-13T08:01:04.331ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
34172025-11-13T08:01:04.331ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a5f0e2f-ff30-4311-a10c-4614d1143044", "content-length": "163", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34182025-11-13T08:01:04.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
34192025-11-13T08:01:04.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "38709ccc-ba2d-4002-8fda-adf611ac6cb8", "content-length": "163", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34202025-11-13T08:01:04.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
34212025-11-13T08:01:04.332ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "005acfd5-ccd6-4212-8d94-c26b6de84f52", "content-length": "166", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34222025-11-13T08:01:04.332ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
34232025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce71c4e2-0a8c-4e17-9420-e9d3502f3f71", "content-length": "166", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34242025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
34252025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2d32f1d-821b-4f64-a512-0526fa737be1", "content-length": "702", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34262025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
34272025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dd3ecec-c0ba-4c82-80cd-28ae01944916", "content-length": "727", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34282025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
34292025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3abadca-9fda-461a-ab6c-0fad1319021b", "content-length": "729", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34302025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
34312025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a2feccb-be11-45c3-8a0e-b31be461090a", "content-length": "728", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34322025-11-13T08:01:04.333ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
34332025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "588a7add-4965-4e48-8412-7720b43492a2", "content-length": "722", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34342025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
34352025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba9fd547-127e-48a9-84be-0cb58d4e4c6a", "content-length": "10", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34362025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
34372025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b4f9e3e-fff2-44db-b800-b9f0c3d51220", "content-length": "144", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34382025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
34392025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "00cd981c-2c12-41e6-841e-2c5ada6c0e84", "content-length": "147", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34402025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
34412025-11-13T08:01:04.334ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da9ea401-8cf0-4e20-aaf9-891e6f2ba02e", "content-length": "109", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34422025-11-13T08:01:04.335ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
34432025-11-13T08:01:04.335ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "07e6cbab-32f4-492c-a3cd-499e78f29bbc", "content-length": "109", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34442025-11-13T08:01:04.335ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
34452025-11-13T08:01:04.335ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "68879652-7260-4ade-8e0b-a6a0fb41f246", "content-length": "163", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34462025-11-13T08:01:04.335ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
34472025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ecaed0f-6878-46fd-8196-fe97bd9bfdbd", "content-length": "163", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34482025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
34492025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ad316ad-bb02-4e44-80aa-ede969828f3a", "content-length": "166", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34502025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
34512025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23ee2c8b-cb5e-4498-94d7-8741de786e93", "content-length": "166", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34522025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
34532025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ddab03d8-3d71-47d3-974a-e50cf23a7873", "content-length": "702", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34542025-11-13T08:01:04.336ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
34552025-11-13T08:01:04.337ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c864ef6-91a9-4c1f-ad57-e040bf8b62f2", "content-length": "727", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34562025-11-13T08:01:04.337ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
34572025-11-13T08:01:04.337ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33b1e800-021f-4cff-8699-3789f845358f", "content-length": "729", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34582025-11-13T08:01:04.337ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
34592025-11-13T08:01:04.337ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14e0177c-af66-486c-b1ca-69011588d5ac", "content-length": "728", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34602025-11-13T08:01:04.337ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34612025-11-13T08:01:04.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
34622025-11-13T08:01:04.339ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34632025-11-13T08:01:04.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
34642025-11-13T08:01:04.341ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34652025-11-13T08:01:04.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
34662025-11-13T08:01:04.343ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34672025-11-13T08:01:04.345ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
34682025-11-13T08:01:04.345ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34692025-11-13T08:01:04.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
34702025-11-13T08:01:04.378ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
34712025-11-13T08:01:04.378ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
34722025-11-13T08:01:04.428ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac5e67b8-d7ae-4a49-844b-2d1573e6114b", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34732025-11-13T08:01:04.431ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
34742025-11-13T08:01:04.431ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
34752025-11-13T08:01:04.431ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
34762025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
34772025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
34782025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
34792025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
34802025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
34812025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e13024f9-c8ee-4890-a8ae-5af872da23e0", "content-length": "64", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34822025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
34832025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "41d2a47d-e61c-4b37-9194-31f107d3d3c1", "content-length": "64", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34842025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8c6997e-9f31-4880-abef-7f8cee6219f3", "content-length": "64", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34852025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
34862025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c2982c5-2939-4434-b48a-84e34bd35928", "content-length": "64", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34872025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
34882025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
34892025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "21c7e6a8-ce6a-4e56-b63a-55b9002feb34", "content-length": "64", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34902025-11-13T08:01:04.431ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
34912025-11-13T08:01:04.468ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8e3c1e8-6ae4-45d7-9807-76ed8e435a67", "content-length": "372702", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34922025-11-13T08:01:04.468ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bedc4628-6850-4cf0-a978-3fe22adf1d75", "content-length": "408759", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34932025-11-13T08:01:04.469ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "44db1643-9a70-4116-a75a-fb639d1160e2", "content-length": "379764", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34942025-11-13T08:01:04.469ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "397c97e4-87f1-4a7c-890e-6552572e19b4", "content-length": "379586", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34952025-11-13T08:01:04.471ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "586d6051-ce63-46e3-90d7-0faebddd5b05", "content-length": "388203", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
34962025-11-13T08:01:04.473ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
34972025-11-13T08:01:04.473ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
34982025-11-13T08:01:04.473ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
34992025-11-13T08:01:04.473ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
35002025-11-13T08:01:04.473ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
35012025-11-13T08:01:04.505ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
35022025-11-13T08:01:04.505ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
35032025-11-13T08:01:04.508ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "71a02efd-9694-4f19-8a10-ef4a45a6c137", "content-length": "122", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
35042025-11-13T08:01:04.508ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
35052025-11-13T08:01:04.599ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
35062025-11-13T08:01:04.599ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
35072025-11-13T08:01:04.600ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc4fd9bd-298f-4166-9fd6-cd67090eb4d3", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
35082025-11-13T08:01:04.600ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
35092025-11-13T08:01:04.602ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
35102025-11-13T08:01:04.602ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
35112025-11-13T08:01:04.602ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d15625ba-4365-4687-a939-f03a0a3a3a8d", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
35122025-11-13T08:01:04.602ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
35132025-11-13T08:01:04.604ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
35142025-11-13T08:01:04.604ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
35152025-11-13T08:01:04.604ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b852aa55-5f88-4ca4-80c7-e0d8cb630292", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:01:04 GMT"} })
35162025-11-13T08:01:04.605ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
35172025-11-13T08:01:04.606ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
35182025-11-13T08:01:04.607ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
35192025-11-13T08:01:04.609ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35202025-11-13T08:01:04.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
35212025-11-13T08:01:04.611ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35222025-11-13T08:01:04.614ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
35232025-11-13T08:01:04.614ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
35242025-11-13T08:01:04.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
35252025-11-13T08:01:04.821ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 55b45047-b092-4620-af64-6b27b076a7c5 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
35262025-11-13T08:01:04.822ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 55b45047-b092-4620-af64-6b27b076a7c5 time_started = 2025-11-13 08:01:04.329832 UTC
35272025-11-13T08:01:04.822ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 13 reason = Dependency
35282025-11-13T08:01:04.822ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
35292025-11-13T08:01:04.822ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 872.951432ms iteration = 4 status = {"collection_id":"55b45047-b092-4620-af64-6b27b076a7c5","time_done":"2025-11-13 08:01:04.606983 UTC","time_started":"2025-11-13 08:01:04.329832 UTC"}
35302025-11-13T08:01:04.822ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
35312025-11-13T08:01:04.824ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35322025-11-13T08:01:04.827ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
35332025-11-13T08:01:04.827ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
35342025-11-13T08:01:04.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
35352025-11-13T08:01:04.829ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
35362025-11-13T08:01:04.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
35372025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 56.389578ms iteration = 13 status = {"Loaded":{"collection_id":"55b45047-b092-4620-af64-6b27b076a7c5","time_loaded":"2025-11-13T08:01:04.822029469Z","time_started":"2025-11-13T08:01:04.329832Z"}}
35382025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 6 reason = Dependency
35392025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 10 reason = Dependency
35402025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 10 reason = Dependency
35412025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35422025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35432025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35442025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35452025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35462025-11-13T08:01:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35472025-11-13T08:01:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35482025-11-13T08:01:04.880ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35492025-11-13T08:01:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35502025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35512025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35522025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35532025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35542025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35552025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
35562025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35572025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35582025-11-13T08:01:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35592025-11-13T08:01:04.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35602025-11-13T08:01:04.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35612025-11-13T08:01:04.885ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35622025-11-13T08:01:04.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35632025-11-13T08:01:04.887ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35642025-11-13T08:01:04.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35652025-11-13T08:01:04.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35662025-11-13T08:01:04.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
35672025-11-13T08:01:04.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35682025-11-13T08:01:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35692025-11-13T08:01:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35702025-11-13T08:01:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
35712025-11-13T08:01:04.890ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
35722025-11-13T08:01:04.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35732025-11-13T08:01:04.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35742025-11-13T08:01:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
35752025-11-13T08:01:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35762025-11-13T08:01:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35772025-11-13T08:01:04.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35782025-11-13T08:01:04.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35792025-11-13T08:01:04.894ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
35802025-11-13T08:01:04.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35812025-11-13T08:01:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35822025-11-13T08:01:04.894ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35832025-11-13T08:01:04.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35842025-11-13T08:01:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35852025-11-13T08:01:04.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35862025-11-13T08:01:04.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35872025-11-13T08:01:04.898ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35882025-11-13T08:01:04.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35892025-11-13T08:01:04.898ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
35902025-11-13T08:01:04.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
35912025-11-13T08:01:04.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35922025-11-13T08:01:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
35932025-11-13T08:01:04.901ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
35942025-11-13T08:01:04.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35952025-11-13T08:01:04.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
35962025-11-13T08:01:04.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35972025-11-13T08:01:04.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
35982025-11-13T08:01:04.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
35992025-11-13T08:01:04.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
36002025-11-13T08:01:04.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36012025-11-13T08:01:04.913ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
36022025-11-13T08:01:04.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
36032025-11-13T08:01:04.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
36042025-11-13T08:01:04.915ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
36052025-11-13T08:01:04.915ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 37.212912ms iteration = 6 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"55b45047-b092-4620-af64-6b27b076a7c5","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
36062025-11-13T08:01:04.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36072025-11-13T08:01:04.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36082025-11-13T08:01:04.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36092025-11-13T08:01:04.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36102025-11-13T08:01:04.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36112025-11-13T08:01:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
36122025-11-13T08:01:04.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36132025-11-13T08:01:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36142025-11-13T08:01:04.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36152025-11-13T08:01:04.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36162025-11-13T08:01:04.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36172025-11-13T08:01:04.963ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
36182025-11-13T08:01:04.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36192025-11-13T08:01:04.963ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36202025-11-13T08:01:04.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36212025-11-13T08:01:04.967ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
36222025-11-13T08:01:04.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36232025-11-13T08:01:04.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36242025-11-13T08:01:04.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36252025-11-13T08:01:04.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36262025-11-13T08:01:04.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36272025-11-13T08:01:04.973ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
36282025-11-13T08:01:04.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36292025-11-13T08:01:04.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36302025-11-13T08:01:04.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36312025-11-13T08:01:04.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36322025-11-13T08:01:04.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36332025-11-13T08:01:04.980ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
36342025-11-13T08:01:04.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36352025-11-13T08:01:04.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36362025-11-13T08:01:04.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36372025-11-13T08:01:04.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36382025-11-13T08:01:04.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36392025-11-13T08:01:04.986ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
36402025-11-13T08:01:04.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36412025-11-13T08:01:04.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36422025-11-13T08:01:05.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36432025-11-13T08:01:05.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36442025-11-13T08:01:05.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36452025-11-13T08:01:05.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36462025-11-13T08:01:05.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36472025-11-13T08:01:05.015ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
36482025-11-13T08:01:05.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36492025-11-13T08:01:05.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36502025-11-13T08:01:05.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36512025-11-13T08:01:05.020ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
36522025-11-13T08:01:05.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36532025-11-13T08:01:05.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36542025-11-13T08:01:05.022ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 143.869218ms iteration = 10 status = {"physical_disks_added":0}
36552025-11-13T08:01:05.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36562025-11-13T08:01:05.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36572025-11-13T08:01:05.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36582025-11-13T08:01:05.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36592025-11-13T08:01:05.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36602025-11-13T08:01:05.029ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
36612025-11-13T08:01:05.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36622025-11-13T08:01:05.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36632025-11-13T08:01:05.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36642025-11-13T08:01:05.034ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
36652025-11-13T08:01:05.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36662025-11-13T08:01:05.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36672025-11-13T08:01:05.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36682025-11-13T08:01:05.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36692025-11-13T08:01:05.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36702025-11-13T08:01:05.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36712025-11-13T08:01:05.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36722025-11-13T08:01:05.042ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
36732025-11-13T08:01:05.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36742025-11-13T08:01:05.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36752025-11-13T08:01:05.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36762025-11-13T08:01:05.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36772025-11-13T08:01:05.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36782025-11-13T08:01:05.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
36792025-11-13T08:01:05.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36802025-11-13T08:01:05.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36812025-11-13T08:01:05.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36822025-11-13T08:01:05.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36832025-11-13T08:01:05.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36842025-11-13T08:01:05.057ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
36852025-11-13T08:01:05.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36862025-11-13T08:01:05.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36872025-11-13T08:01:05.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36882025-11-13T08:01:05.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36892025-11-13T08:01:05.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36902025-11-13T08:01:05.063ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
36912025-11-13T08:01:05.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36922025-11-13T08:01:05.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36932025-11-13T08:01:05.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36942025-11-13T08:01:05.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36952025-11-13T08:01:05.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36962025-11-13T08:01:05.068ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
36972025-11-13T08:01:05.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
36982025-11-13T08:01:05.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
36992025-11-13T08:01:05.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37002025-11-13T08:01:05.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37012025-11-13T08:01:05.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37022025-11-13T08:01:05.074ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
37032025-11-13T08:01:05.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37042025-11-13T08:01:05.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37052025-11-13T08:01:05.107ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
37062025-11-13T08:01:05.107ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
37072025-11-13T08:01:05.107ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 7baf069f-92b5-4865-a108-b54bd2b62d5b parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
37082025-11-13T08:01:05.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37092025-11-13T08:01:05.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37102025-11-13T08:01:05.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37112025-11-13T08:01:05.112ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
37122025-11-13T08:01:05.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
37132025-11-13T08:01:05.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
37142025-11-13T08:01:05.115ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
37152025-11-13T08:01:05.115ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
37162025-11-13T08:01:05.115ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 236.752365ms iteration = 10 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
37172025-11-13T08:01:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 295 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 364d5022-761b-48fe-8235-b560d551b715 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
37182025-11-13T08:01:07.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 26 reason = Timeout
37192025-11-13T08:01:07.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37202025-11-13T08:01:07.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37212025-11-13T08:01:07.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37222025-11-13T08:01:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
37232025-11-13T08:01:07.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37242025-11-13T08:01:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37252025-11-13T08:01:07.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
37262025-11-13T08:01:07.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.044988ms iteration = 26 status = {"config_updated":false}
37272025-11-13T08:01:12.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 14 reason = Timeout
37282025-11-13T08:01:12.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 27 reason = Timeout
37292025-11-13T08:01:12.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37302025-11-13T08:01:12.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37312025-11-13T08:01:12.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37322025-11-13T08:01:12.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37332025-11-13T08:01:12.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37342025-11-13T08:01:12.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37352025-11-13T08:01:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
37362025-11-13T08:01:12.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37372025-11-13T08:01:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
37382025-11-13T08:01:12.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37392025-11-13T08:01:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37402025-11-13T08:01:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37412025-11-13T08:01:12.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37422025-11-13T08:01:12.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37432025-11-13T08:01:12.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
37442025-11-13T08:01:12.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.300803ms iteration = 27 status = {"config_updated":false}
37452025-11-13T08:01:12.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37462025-11-13T08:01:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
37472025-11-13T08:01:12.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
37482025-11-13T08:01:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
37492025-11-13T08:01:12.641ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
37502025-11-13T08:01:12.641ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.025483ms iteration = 14 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
37512025-11-13T08:01:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 14 reason = Timeout
37522025-11-13T08:01:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37532025-11-13T08:01:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37542025-11-13T08:01:12.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
37552025-11-13T08:01:12.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
37562025-11-13T08:01:12.678ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.715942ms iteration = 14 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
37572025-11-13T08:01:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 320 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 86c47110-71d0-4450-9a49-1ef6531af281 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
37582025-11-13T08:01:17.617ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 14 reason = Timeout
37592025-11-13T08:01:17.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 28 reason = Timeout
37602025-11-13T08:01:17.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37612025-11-13T08:01:17.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
37622025-11-13T08:01:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37632025-11-13T08:01:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
37642025-11-13T08:01:17.620ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37652025-11-13T08:01:17.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37662025-11-13T08:01:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
37672025-11-13T08:01:17.624ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
37682025-11-13T08:01:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
37692025-11-13T08:01:17.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
37702025-11-13T08:01:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
37712025-11-13T08:01:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
37722025-11-13T08:01:17.626ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 55b45047-b092-4620-af64-6b27b076a7c5 original_time_started = 2025-11-13 08:01:04.329832 UTC
37732025-11-13T08:01:17.626ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.745412ms iteration = 14 status = {"Loaded":{"collection_id":"55b45047-b092-4620-af64-6b27b076a7c5","time_loaded":"2025-11-13T08:01:17.617897704Z","time_started":"2025-11-13T08:01:04.329832Z"}}
37742025-11-13T08:01:17.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
37752025-11-13T08:01:17.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.896785ms iteration = 28 status = {"config_updated":false}
37762025-11-13T08:01:17.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 10 reason = Timeout
37772025-11-13T08:01:17.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 10 reason = Timeout
37782025-11-13T08:01:17.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
37792025-11-13T08:01:17.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
37802025-11-13T08:01:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
37812025-11-13T08:01:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
37822025-11-13T08:01:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
37832025-11-13T08:01:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
37842025-11-13T08:01:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37852025-11-13T08:01:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37862025-11-13T08:01:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
37872025-11-13T08:01:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
37882025-11-13T08:01:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
37892025-11-13T08:01:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
37902025-11-13T08:01:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
37912025-11-13T08:01:17.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
37922025-11-13T08:01:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.483517ms iteration = 10 status = "NoSitrep"
37932025-11-13T08:01:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
37942025-11-13T08:01:17.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
37952025-11-13T08:01:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37962025-11-13T08:01:17.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
37972025-11-13T08:01:17.657ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
37982025-11-13T08:01:17.657ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.820593ms iteration = 10 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
37992025-11-13T08:01:22.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 15 reason = Timeout
38002025-11-13T08:01:22.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 29 reason = Timeout
38012025-11-13T08:01:22.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38022025-11-13T08:01:22.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38032025-11-13T08:01:22.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38042025-11-13T08:01:22.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38052025-11-13T08:01:22.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38062025-11-13T08:01:22.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38072025-11-13T08:01:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38082025-11-13T08:01:22.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38092025-11-13T08:01:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
38102025-11-13T08:01:22.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38112025-11-13T08:01:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38122025-11-13T08:01:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38132025-11-13T08:01:22.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
38142025-11-13T08:01:22.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.342465ms iteration = 29 status = {"config_updated":false}
38152025-11-13T08:01:22.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38162025-11-13T08:01:22.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38172025-11-13T08:01:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38182025-11-13T08:01:22.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
38192025-11-13T08:01:22.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38202025-11-13T08:01:22.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38212025-11-13T08:01:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 15 reason = Timeout
38222025-11-13T08:01:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38232025-11-13T08:01:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38242025-11-13T08:01:22.645ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
38252025-11-13T08:01:22.645ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.243555ms iteration = 15 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
38262025-11-13T08:01:22.676ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
38272025-11-13T08:01:22.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
38282025-11-13T08:01:22.709ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 67.001237ms iteration = 15 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
38292025-11-13T08:01:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 292 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 108dd78a-8e44-4fe6-9994-87f0a443721b response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
38302025-11-13T08:01:27.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 30 reason = Timeout
38312025-11-13T08:01:27.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38322025-11-13T08:01:27.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38332025-11-13T08:01:27.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38342025-11-13T08:01:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
38352025-11-13T08:01:27.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38362025-11-13T08:01:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38372025-11-13T08:01:27.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
38382025-11-13T08:01:27.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.10767ms iteration = 30 status = {"config_updated":false}
38392025-11-13T08:01:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 6 reason = Timeout
38402025-11-13T08:01:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 6 reason = Timeout
38412025-11-13T08:01:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
38422025-11-13T08:01:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 6 reason = Timeout
38432025-11-13T08:01:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
38442025-11-13T08:01:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
38452025-11-13T08:01:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
38462025-11-13T08:01:32.611ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6100, tv_nsec: 963792267 } })
38472025-11-13T08:01:32.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6100, tv_nsec: 963795458 } })
38482025-11-13T08:01:32.611ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
38492025-11-13T08:01:32.613ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 5.820134ms iteration = 6 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
38502025-11-13T08:01:32.617ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 9 reason = Timeout
38512025-11-13T08:01:32.617ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 15 reason = Timeout
38522025-11-13T08:01:32.617ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38532025-11-13T08:01:32.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
38542025-11-13T08:01:32.617ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 11 reason = Timeout
38552025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 16 reason = Timeout
38562025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38572025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 31 reason = Timeout
38582025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38592025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38602025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
38612025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38622025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38632025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38642025-11-13T08:01:32.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38652025-11-13T08:01:32.620ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38662025-11-13T08:01:32.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38672025-11-13T08:01:32.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38682025-11-13T08:01:32.621ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38692025-11-13T08:01:32.623ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38702025-11-13T08:01:32.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38712025-11-13T08:01:32.624ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38722025-11-13T08:01:32.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38732025-11-13T08:01:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
38742025-11-13T08:01:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
38752025-11-13T08:01:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
38762025-11-13T08:01:32.627ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38772025-11-13T08:01:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
38782025-11-13T08:01:32.627ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38792025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
38802025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38812025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38822025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
38832025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
38842025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38852025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38862025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
38872025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
38882025-11-13T08:01:32.628ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 10.874253ms iteration = 31 status = {"config_updated":false}
38892025-11-13T08:01:32.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38902025-11-13T08:01:32.630ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 55b45047-b092-4620-af64-6b27b076a7c5 original_time_started = 2025-11-13 08:01:04.329832 UTC
38912025-11-13T08:01:32.630ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.790462ms iteration = 15 status = {"Loaded":{"collection_id":"55b45047-b092-4620-af64-6b27b076a7c5","time_loaded":"2025-11-13T08:01:32.617936841Z","time_started":"2025-11-13T08:01:04.329832Z"}}
38922025-11-13T08:01:32.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
38932025-11-13T08:01:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
38942025-11-13T08:01:32.632ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
38952025-11-13T08:01:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
38962025-11-13T08:01:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
38972025-11-13T08:01:32.634ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
38982025-11-13T08:01:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
38992025-11-13T08:01:32.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39002025-11-13T08:01:32.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
39012025-11-13T08:01:32.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
39022025-11-13T08:01:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
39032025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 6 reason = Timeout
39042025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 6 reason = Timeout
39052025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39062025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39072025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 6 reason = Timeout
39082025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39092025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 9 reason = Timeout
39102025-11-13T08:01:32.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
39112025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 16 reason = Timeout
39122025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39132025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
39142025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 6 reason = Timeout
39152025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
39162025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39172025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 6 reason = Timeout
39182025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39192025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 6 reason = Timeout
39202025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39212025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
39222025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39232025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39242025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39252025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
39262025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39272025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
39282025-11-13T08:01:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
39292025-11-13T08:01:32.643ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39302025-11-13T08:01:32.646ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39312025-11-13T08:01:32.646ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39322025-11-13T08:01:32.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39332025-11-13T08:01:32.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39342025-11-13T08:01:32.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
39352025-11-13T08:01:32.646ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
39362025-11-13T08:01:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
39372025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 6 reason = Timeout
39382025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 6 reason = Timeout
39392025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
39402025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 6 reason = Timeout
39412025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
39422025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 6 reason = Timeout
39432025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
39442025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 11 reason = Timeout
39452025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 11 reason = Timeout
39462025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
39472025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
39482025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 6 reason = Timeout
39492025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
39502025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 6 reason = Timeout
39512025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
39522025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
39532025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
39542025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
39552025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
39562025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
39572025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
39582025-11-13T08:01:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
39592025-11-13T08:01:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6101, tv_nsec: 1516599 } })
39602025-11-13T08:01:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
39612025-11-13T08:01:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
39622025-11-13T08:01:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
39632025-11-13T08:01:32.650ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
39642025-11-13T08:01:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
39652025-11-13T08:01:32.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
39662025-11-13T08:01:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
39672025-11-13T08:01:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
39682025-11-13T08:01:32.653ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6101, tv_nsec: 6320644 } })
39692025-11-13T08:01:32.654ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
39702025-11-13T08:01:32.678ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39712025-11-13T08:01:32.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39722025-11-13T08:01:32.689ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
39732025-11-13T08:01:32.689ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 71.640269ms iteration = 16 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
39742025-11-13T08:01:32.698ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39752025-11-13T08:01:32.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39762025-11-13T08:01:32.700ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39772025-11-13T08:01:32.704ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
39782025-11-13T08:01:32.704ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
39792025-11-13T08:01:32.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
39802025-11-13T08:01:32.720ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39812025-11-13T08:01:32.721ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
39822025-11-13T08:01:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
39832025-11-13T08:01:32.724ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
39842025-11-13T08:01:32.724ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
39852025-11-13T08:01:32.724ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39862025-11-13T08:01:32.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
39872025-11-13T08:01:32.725ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
39882025-11-13T08:01:32.725ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
39892025-11-13T08:01:32.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
39902025-11-13T08:01:32.726ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
39912025-11-13T08:01:32.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
39922025-11-13T08:01:32.728ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
39932025-11-13T08:01:32.728ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
39942025-11-13T08:01:32.728ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
39952025-11-13T08:01:32.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
39962025-11-13T08:01:32.728ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 85.98131ms iteration = 6 status = 0
39972025-11-13T08:01:32.731ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39982025-11-13T08:01:32.732ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
39992025-11-13T08:01:32.732ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 114.702465ms iteration = 9 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
40002025-11-13T08:01:32.732ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40012025-11-13T08:01:32.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40022025-11-13T08:01:32.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40032025-11-13T08:01:32.735ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
40042025-11-13T08:01:32.735ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
40052025-11-13T08:01:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
40062025-11-13T08:01:32.735ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 87.147533ms iteration = 6 status = {"errors":[],"finish_invoked_ok":[]}
40072025-11-13T08:01:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
40082025-11-13T08:01:32.736ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 87.961489ms iteration = 6 status = {"errors":[],"garbage_collect_requested":[]}
40092025-11-13T08:01:32.737ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 89.676463ms iteration = 11 status = "NoSitrep"
40102025-11-13T08:01:32.738ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40112025-11-13T08:01:32.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40122025-11-13T08:01:32.738ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
40132025-11-13T08:01:32.738ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40142025-11-13T08:01:32.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
40152025-11-13T08:01:32.742ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40162025-11-13T08:01:32.742ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40172025-11-13T08:01:32.742ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 94.736563ms iteration = 6 status = {"errors":[],"requests_created_ok":[]}
40182025-11-13T08:01:32.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40192025-11-13T08:01:32.744ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
40202025-11-13T08:01:32.744ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 102.116148ms iteration = 6 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
40212025-11-13T08:01:32.747ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
40222025-11-13T08:01:32.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
40232025-11-13T08:01:32.756ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 108.519804ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40242025-11-13T08:01:32.773ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40252025-11-13T08:01:32.775ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
40262025-11-13T08:01:32.775ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
40272025-11-13T08:01:32.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
40282025-11-13T08:01:32.782ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 140.258507ms iteration = 6 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
40292025-11-13T08:01:32.786ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
40302025-11-13T08:01:32.787ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6e6e0af-99a1-4550-aa42-0ea54e3db815", "content-length": "78", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40312025-11-13T08:01:32.787ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
40322025-11-13T08:01:32.787ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
40332025-11-13T08:01:32.787ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
40342025-11-13T08:01:32.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
40352025-11-13T08:01:32.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
40362025-11-13T08:01:32.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
40372025-11-13T08:01:32.789ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
40382025-11-13T08:01:32.789ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
40392025-11-13T08:01:32.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 147.639743ms iteration = 16 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
40402025-11-13T08:01:32.790ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
40412025-11-13T08:01:32.791ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
40422025-11-13T08:01:32.791ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
40432025-11-13T08:01:32.791ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 148.720534ms iteration = 6 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
40442025-11-13T08:01:32.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
40452025-11-13T08:01:32.792ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
40462025-11-13T08:01:32.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
40472025-11-13T08:01:32.793ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
40482025-11-13T08:01:32.793ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
40492025-11-13T08:01:32.793ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
40502025-11-13T08:01:32.793ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
40512025-11-13T08:01:32.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
40522025-11-13T08:01:32.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
40532025-11-13T08:01:32.796ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40542025-11-13T08:01:32.797ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40552025-11-13T08:01:32.797ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
40562025-11-13T08:01:32.798ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
40572025-11-13T08:01:32.798ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 150.526779ms iteration = 11 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
40582025-11-13T08:01:32.800ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
40592025-11-13T08:01:32.800ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
40602025-11-13T08:01:32.800ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40612025-11-13T08:01:32.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
40622025-11-13T08:01:32.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
40632025-11-13T08:01:32.801ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40642025-11-13T08:01:32.801ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a83d3ab-085a-4a45-b0b5-088353fa3fbc", "content-length": "26", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40652025-11-13T08:01:32.801ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40662025-11-13T08:01:32.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
40672025-11-13T08:01:32.802ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
40682025-11-13T08:01:32.841ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
40692025-11-13T08:01:32.841ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
40702025-11-13T08:01:32.841ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4373aa5-102a-4b50-bde1-6f46460c744c", "content-length": "26", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40712025-11-13T08:01:32.841ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
40722025-11-13T08:01:32.843ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
40732025-11-13T08:01:32.843ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 225.646816ms iteration = 11 status = {"physical_disks_added":0}
40742025-11-13T08:01:32.845ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
40752025-11-13T08:01:32.845ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
40762025-11-13T08:01:32.846ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eea70ea-28d9-44a7-8dea-023246da08a3", "content-length": "81", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40772025-11-13T08:01:32.849ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
40782025-11-13T08:01:32.849ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
40792025-11-13T08:01:32.849ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3763863b-d0e0-4502-ab48-f3cb1a5415ce", "content-length": "81", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40802025-11-13T08:01:32.852ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
40812025-11-13T08:01:32.852ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
40822025-11-13T08:01:32.852ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "902a25f3-72a7-42cf-a355-276dcd63333d", "content-length": "81", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40832025-11-13T08:01:32.854ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 206.327097ms iteration = 6 status = {"disabled":false,"errors":[],"sps":[]}
40842025-11-13T08:01:32.854ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 206.348666ms iteration = 6 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
40852025-11-13T08:01:32.865ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
40862025-11-13T08:01:32.866ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 258.88403ms iteration = 6 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
40872025-11-13T08:01:32.873ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
40882025-11-13T08:01:32.874ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6af842c2-b450-492f-890c-787d6283dda6", "content-length": "691", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40892025-11-13T08:01:32.874ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
40902025-11-13T08:01:32.874ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
40912025-11-13T08:01:32.874ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
40922025-11-13T08:01:32.875ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 233.744145ms iteration = 6 status = {}
40932025-11-13T08:01:32.899ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6101, tv_nsec: 252149827 } })
40942025-11-13T08:01:32.926ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 319.141874ms iteration = 6 status = {}
40952025-11-13T08:01:32.960ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
40962025-11-13T08:01:32.960ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
40972025-11-13T08:01:32.961ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f774f32-e433-4212-a6eb-fda0505f8db1", "content-length": "372", "date": "Thu, 13 Nov 2025 08:01:32 GMT"} })
40982025-11-13T08:01:32.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
40992025-11-13T08:01:32.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41002025-11-13T08:01:32.996ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41012025-11-13T08:01:32.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41022025-11-13T08:01:32.998ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
41032025-11-13T08:01:32.998ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41042025-11-13T08:01:32.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41052025-11-13T08:01:33.011ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
41062025-11-13T08:01:33.011ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
41072025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
41082025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
41092025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41102025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41112025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41122025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41132025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
41142025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
41152025-11-13T08:01:33.012ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
41162025-11-13T08:01:33.012ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41172025-11-13T08:01:33.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41182025-11-13T08:01:33.021ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
41192025-11-13T08:01:33.021ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
41202025-11-13T08:01:33.023ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
41212025-11-13T08:01:33.023ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
41222025-11-13T08:01:33.025ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8e9ba5d4-408c-4036-8105-4d3a6cc193a6", "date": "Thu, 13 Nov 2025 08:01:33 GMT"} })
41232025-11-13T08:01:33.025ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41242025-11-13T08:01:33.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41252025-11-13T08:01:33.028ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41262025-11-13T08:01:33.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41272025-11-13T08:01:33.031ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
41282025-11-13T08:01:33.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
41292025-11-13T08:01:33.035ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
41302025-11-13T08:01:33.035ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
41312025-11-13T08:01:33.035ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "52c6fd28-1a6d-4870-b3b2-2b1d8e99f85b", "date": "Thu, 13 Nov 2025 08:01:33 GMT"} })
41322025-11-13T08:01:33.037ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 395.027626ms iteration = 9 status = {}
41332025-11-13T08:01:33.240ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4611ff3b-2634-43fc-8395-6c59b85c9643", "date": "Thu, 13 Nov 2025 08:01:33 GMT"} })
41342025-11-13T08:01:33.240ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
41352025-11-13T08:01:33.240ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
41362025-11-13T08:01:33.240ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "09b22cbf-50f1-4455-93ce-9641ae2942af", "content-length": "682", "date": "Thu, 13 Nov 2025 08:01:33 GMT"} })
41372025-11-13T08:01:33.240ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41382025-11-13T08:01:33.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41392025-11-13T08:01:33.244ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41402025-11-13T08:01:33.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41412025-11-13T08:01:33.247ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41422025-11-13T08:01:33.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41432025-11-13T08:01:33.249ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
41442025-11-13T08:01:33.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
41452025-11-13T08:01:33.256ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 614.583792ms iteration = 6 status = {}
41462025-11-13T08:01:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 313 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 3b6e16db-46eb-485b-9578-2ea9bb9984d0 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
41472025-11-13T08:01:37.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 32 reason = Timeout
41482025-11-13T08:01:37.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
41492025-11-13T08:01:37.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
41502025-11-13T08:01:37.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41512025-11-13T08:01:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
41522025-11-13T08:01:37.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
41532025-11-13T08:01:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
41542025-11-13T08:01:37.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
41552025-11-13T08:01:37.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.12718ms iteration = 32 status = {"config_updated":false}
41562025-11-13T08:01:42.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 17 reason = Timeout
41572025-11-13T08:01:42.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 33 reason = Timeout
41582025-11-13T08:01:42.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
41592025-11-13T08:01:42.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41602025-11-13T08:01:42.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
41612025-11-13T08:01:42.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
41622025-11-13T08:01:42.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41632025-11-13T08:01:42.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41642025-11-13T08:01:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
41652025-11-13T08:01:42.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
41662025-11-13T08:01:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
41672025-11-13T08:01:42.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41682025-11-13T08:01:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
41692025-11-13T08:01:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
41702025-11-13T08:01:42.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
41712025-11-13T08:01:42.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.705291ms iteration = 33 status = {"config_updated":false}
41722025-11-13T08:01:42.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41732025-11-13T08:01:42.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
41742025-11-13T08:01:42.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41752025-11-13T08:01:42.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
41762025-11-13T08:01:42.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
41772025-11-13T08:01:42.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
41782025-11-13T08:01:42.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 17 reason = Timeout
41792025-11-13T08:01:42.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41802025-11-13T08:01:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41812025-11-13T08:01:42.646ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
41822025-11-13T08:01:42.646ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.577811ms iteration = 17 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
41832025-11-13T08:01:42.676ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
41842025-11-13T08:01:42.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
41852025-11-13T08:01:42.678ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.396146ms iteration = 17 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
41862025-11-13T08:01:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 320 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 18f4bfef-6100-4789-9652-591536cefbb2 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
41872025-11-13T08:01:47.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 16 reason = Timeout
41882025-11-13T08:01:47.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 34 reason = Timeout
41892025-11-13T08:01:47.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
41902025-11-13T08:01:47.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
41912025-11-13T08:01:47.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
41922025-11-13T08:01:47.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
41932025-11-13T08:01:47.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41942025-11-13T08:01:47.620ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41952025-11-13T08:01:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
41962025-11-13T08:01:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
41972025-11-13T08:01:47.623ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
41982025-11-13T08:01:47.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
41992025-11-13T08:01:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
42002025-11-13T08:01:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42012025-11-13T08:01:47.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
42022025-11-13T08:01:47.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.57111ms iteration = 34 status = {"config_updated":false}
42032025-11-13T08:01:47.625ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 55b45047-b092-4620-af64-6b27b076a7c5 original_time_started = 2025-11-13 08:01:04.329832 UTC
42042025-11-13T08:01:47.625ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.1161ms iteration = 16 status = {"Loaded":{"collection_id":"55b45047-b092-4620-af64-6b27b076a7c5","time_loaded":"2025-11-13T08:01:47.618379388Z","time_started":"2025-11-13T08:01:04.329832Z"}}
42052025-11-13T08:01:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 12 reason = Timeout
42062025-11-13T08:01:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 12 reason = Timeout
42072025-11-13T08:01:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
42082025-11-13T08:01:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
42092025-11-13T08:01:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
42102025-11-13T08:01:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
42112025-11-13T08:01:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
42122025-11-13T08:01:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
42132025-11-13T08:01:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42142025-11-13T08:01:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42152025-11-13T08:01:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
42162025-11-13T08:01:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
42172025-11-13T08:01:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
42182025-11-13T08:01:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
42192025-11-13T08:01:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.827555ms iteration = 12 status = "NoSitrep"
42202025-11-13T08:01:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
42212025-11-13T08:01:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
42222025-11-13T08:01:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
42232025-11-13T08:01:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
42242025-11-13T08:01:47.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42252025-11-13T08:01:47.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
42262025-11-13T08:01:47.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
42272025-11-13T08:01:47.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.522817ms iteration = 12 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
42282025-11-13T08:01:52.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 18 reason = Timeout
42292025-11-13T08:01:52.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 35 reason = Timeout
42302025-11-13T08:01:52.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42312025-11-13T08:01:52.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42322025-11-13T08:01:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42332025-11-13T08:01:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
42342025-11-13T08:01:52.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42352025-11-13T08:01:52.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42362025-11-13T08:01:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
42372025-11-13T08:01:52.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42382025-11-13T08:01:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
42392025-11-13T08:01:52.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42402025-11-13T08:01:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
42412025-11-13T08:01:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42422025-11-13T08:01:52.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
42432025-11-13T08:01:52.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.442236ms iteration = 35 status = {"config_updated":false}
42442025-11-13T08:01:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42452025-11-13T08:01:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
42462025-11-13T08:01:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42472025-11-13T08:01:52.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
42482025-11-13T08:01:52.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
42492025-11-13T08:01:52.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
42502025-11-13T08:01:52.642ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
42512025-11-13T08:01:52.642ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.007861ms iteration = 18 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
42522025-11-13T08:01:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 18 reason = Timeout
42532025-11-13T08:01:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42542025-11-13T08:01:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42552025-11-13T08:01:52.677ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
42562025-11-13T08:01:52.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
42572025-11-13T08:01:52.711ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.333403ms iteration = 18 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
42582025-11-13T08:01:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 369 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 6cc1ee7d-bc48-4b21-9c87-234f494523b6 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
42592025-11-13T08:01:57.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 36 reason = Timeout
42602025-11-13T08:01:57.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42612025-11-13T08:01:57.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42622025-11-13T08:01:57.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42632025-11-13T08:01:57.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
42642025-11-13T08:01:57.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
42652025-11-13T08:01:57.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
42662025-11-13T08:01:57.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
42672025-11-13T08:01:57.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.936917ms iteration = 36 status = {"config_updated":false}
42682025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 4 reason = Timeout
42692025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 4 reason = Timeout
42702025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 7 reason = Timeout
42712025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
42722025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 7 reason = Timeout
42732025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
42742025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 7 reason = Timeout
42752025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
42762025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 6 reason = Timeout
42772025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 6 reason = Timeout
42782025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 4 reason = Timeout
42792025-11-13T08:02:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:52:02.607471463 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
42802025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
42812025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 4 reason = Timeout
42822025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
42832025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
42842025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 4 reason = Timeout
42852025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 4 reason = Timeout
42862025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
42872025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
42882025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 5 reason = Timeout
42892025-11-13T08:02:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
42902025-11-13T08:02:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
42912025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
42922025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
42932025-11-13T08:02:02.608ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = ExternalNetworking step id = Ensure
42942025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
42952025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
42962025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
42972025-11-13T08:02:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6130, tv_nsec: 960499471 } })
42982025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
42992025-11-13T08:02:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6130, tv_nsec: 960613053 } })
43002025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43012025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
43022025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
43032025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.465438ms iteration = 4 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
43042025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
43052025-11-13T08:02:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.413328ms iteration = 4 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
43062025-11-13T08:02:02.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
43072025-11-13T08:02:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
43082025-11-13T08:02:02.611ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6130, tv_nsec: 964051421 } })
43092025-11-13T08:02:02.612ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
43102025-11-13T08:02:02.615ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
43112025-11-13T08:02:02.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
43122025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 3 reason = Timeout
43132025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 12 reason = Timeout
43142025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
43152025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 11 reason = Timeout
43162025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43172025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43182025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 4 reason = Timeout
43192025-11-13T08:02:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
43202025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 10 reason = Timeout
43212025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
43222025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43232025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 19 reason = Timeout
43242025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
43252025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 17 reason = Timeout
43262025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 37 reason = Timeout
43272025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
43282025-11-13T08:02:02.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
43292025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
43302025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43312025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43322025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
43332025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
43342025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43352025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
43362025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
43372025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43382025-11-13T08:02:02.619ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:52:02.607471463 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
43392025-11-13T08:02:02.619ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 12.464075ms iteration = 4 status = {"expiration":"2025-11-13T07:52:02.607471463Z","pruned":[]}
43402025-11-13T08:02:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43412025-11-13T08:02:02.620ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43422025-11-13T08:02:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43432025-11-13T08:02:02.620ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43442025-11-13T08:02:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43452025-11-13T08:02:02.620ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43462025-11-13T08:02:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43472025-11-13T08:02:02.622ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43482025-11-13T08:02:02.624ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43492025-11-13T08:02:02.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43502025-11-13T08:02:02.626ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43512025-11-13T08:02:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
43522025-11-13T08:02:02.626ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
43532025-11-13T08:02:02.626ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 19.260309ms iteration = 7 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
43542025-11-13T08:02:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
43552025-11-13T08:02:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
43562025-11-13T08:02:02.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
43572025-11-13T08:02:02.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43582025-11-13T08:02:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
43592025-11-13T08:02:02.629ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43602025-11-13T08:02:02.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
43612025-11-13T08:02:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
43622025-11-13T08:02:02.631ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43632025-11-13T08:02:02.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43642025-11-13T08:02:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
43652025-11-13T08:02:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43662025-11-13T08:02:02.633ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43672025-11-13T08:02:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
43682025-11-13T08:02:02.635ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
43692025-11-13T08:02:02.635ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
43702025-11-13T08:02:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
43712025-11-13T08:02:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
43722025-11-13T08:02:02.636ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
43732025-11-13T08:02:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
43742025-11-13T08:02:02.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
43752025-11-13T08:02:02.636ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
43762025-11-13T08:02:02.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
43772025-11-13T08:02:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
43782025-11-13T08:02:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
43792025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43802025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
43812025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
43822025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
43832025-11-13T08:02:02.643ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6130, tv_nsec: 993041461 } })
43842025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 7 reason = Timeout
43852025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 4 reason = Timeout
43862025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
43872025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
43882025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
43892025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 7 reason = Timeout
43902025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
43912025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 4 reason = Timeout
43922025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
43932025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 10 reason = Timeout
43942025-11-13T08:02:02.643ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
43952025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
43962025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 7 reason = Timeout
43972025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
43982025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 7 reason = Timeout
43992025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44002025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 7 reason = Timeout
44012025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
44022025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 19 reason = Timeout
44032025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44042025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 7 reason = Timeout
44052025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
44062025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44072025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
44082025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44092025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
44102025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
44112025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
44122025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
44132025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
44142025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44152025-11-13T08:02:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
44162025-11-13T08:02:02.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44172025-11-13T08:02:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
44182025-11-13T08:02:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 4 reason = Timeout
44192025-11-13T08:02:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
44202025-11-13T08:02:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
44212025-11-13T08:02:02.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44222025-11-13T08:02:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44232025-11-13T08:02:02.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 7 reason = Timeout
44242025-11-13T08:02:02.647ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 4 reason = Timeout
44252025-11-13T08:02:02.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
44262025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
44272025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 7 reason = Timeout
44282025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
44292025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 7 reason = Timeout
44302025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 7 reason = Timeout
44312025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
44322025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 7 reason = Timeout
44332025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
44342025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 7 reason = Timeout
44352025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
44362025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 13 reason = Timeout
44372025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
44382025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 13 reason = Timeout
44392025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
44402025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 4 reason = Timeout
44412025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
44422025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
44432025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
44442025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
44452025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
44462025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
44472025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
44482025-11-13T08:02:02.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6131, tv_nsec: 938666 } })
44492025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
44502025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
44512025-11-13T08:02:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
44522025-11-13T08:02:02.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
44532025-11-13T08:02:02.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
44542025-11-13T08:02:02.649ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 4 reason = Timeout
44552025-11-13T08:02:02.649ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
44562025-11-13T08:02:02.649ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
44572025-11-13T08:02:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
44582025-11-13T08:02:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
44592025-11-13T08:02:02.651ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
44602025-11-13T08:02:02.654ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
44612025-11-13T08:02:02.655ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 55b45047-b092-4620-af64-6b27b076a7c5 original_time_started = 2025-11-13 08:01:04.329832 UTC
44622025-11-13T08:02:02.655ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 36.804654ms iteration = 17 status = {"Loaded":{"collection_id":"55b45047-b092-4620-af64-6b27b076a7c5","time_loaded":"2025-11-13T08:02:02.618645922Z","time_started":"2025-11-13T08:01:04.329832Z"}}
44632025-11-13T08:02:02.656ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
44642025-11-13T08:02:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
44652025-11-13T08:02:02.659ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
44662025-11-13T08:02:02.662ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44672025-11-13T08:02:02.664ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
44682025-11-13T08:02:02.666ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
44692025-11-13T08:02:02.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
44702025-11-13T08:02:02.666ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
44712025-11-13T08:02:02.666ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
44722025-11-13T08:02:02.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
44732025-11-13T08:02:02.667ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44742025-11-13T08:02:02.669ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44752025-11-13T08:02:02.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
44762025-11-13T08:02:02.672ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
44772025-11-13T08:02:02.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
44782025-11-13T08:02:02.675ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 29.47213ms iteration = 4 status = {"errors":[],"found_port_ok":[]}
44792025-11-13T08:02:02.676ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
44802025-11-13T08:02:02.676ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
44812025-11-13T08:02:02.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
44822025-11-13T08:02:02.678ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
44832025-11-13T08:02:02.685ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 37.249492ms iteration = 7 status = {"errors":[],"garbage_collect_requested":[]}
44842025-11-13T08:02:02.686ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
44852025-11-13T08:02:02.690ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
44862025-11-13T08:02:02.700ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
44872025-11-13T08:02:02.702ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
44882025-11-13T08:02:02.706ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
44892025-11-13T08:02:02.708ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
44902025-11-13T08:02:02.711ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
44912025-11-13T08:02:02.711ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = SledList step id = Fetch
44922025-11-13T08:02:02.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
44932025-11-13T08:02:02.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
44942025-11-13T08:02:02.712ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44952025-11-13T08:02:02.716ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
44962025-11-13T08:02:02.716ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 68.12406ms iteration = 4 status = {"by_rx":{},"error":null}
44972025-11-13T08:02:02.718ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44982025-11-13T08:02:02.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
44992025-11-13T08:02:02.718ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 70.785402ms iteration = 7 status = {"errors":[],"finish_invoked_ok":[]}
45002025-11-13T08:02:02.719ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
45012025-11-13T08:02:02.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
45022025-11-13T08:02:02.722ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
45032025-11-13T08:02:02.722ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45042025-11-13T08:02:02.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45052025-11-13T08:02:02.725ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45062025-11-13T08:02:02.728ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
45072025-11-13T08:02:02.729ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
45082025-11-13T08:02:02.729ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 110.442171ms iteration = 4 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
45092025-11-13T08:02:02.730ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
45102025-11-13T08:02:02.730ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45112025-11-13T08:02:02.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45122025-11-13T08:02:02.733ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
45132025-11-13T08:02:02.733ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
45142025-11-13T08:02:02.733ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
45152025-11-13T08:02:02.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
45162025-11-13T08:02:02.745ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45172025-11-13T08:02:02.749ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
45182025-11-13T08:02:02.749ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
45192025-11-13T08:02:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
45202025-11-13T08:02:02.751ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45212025-11-13T08:02:02.751ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45222025-11-13T08:02:02.752ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45232025-11-13T08:02:02.752ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45242025-11-13T08:02:02.753ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
45252025-11-13T08:02:02.753ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
45262025-11-13T08:02:02.753ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
45272025-11-13T08:02:02.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45282025-11-13T08:02:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45292025-11-13T08:02:02.755ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45302025-11-13T08:02:02.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
45312025-11-13T08:02:02.756ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
45322025-11-13T08:02:02.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
45332025-11-13T08:02:02.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45342025-11-13T08:02:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
45352025-11-13T08:02:02.757ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45362025-11-13T08:02:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45372025-11-13T08:02:02.757ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 115.96956ms iteration = 7 status = 0
45382025-11-13T08:02:02.758ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45392025-11-13T08:02:02.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45402025-11-13T08:02:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
45412025-11-13T08:02:02.760ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
45422025-11-13T08:02:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
45432025-11-13T08:02:02.760ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
45442025-11-13T08:02:02.760ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
45452025-11-13T08:02:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
45462025-11-13T08:02:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
45472025-11-13T08:02:02.761ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
45482025-11-13T08:02:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
45492025-11-13T08:02:02.761ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
45502025-11-13T08:02:02.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
45512025-11-13T08:02:02.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45522025-11-13T08:02:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
45532025-11-13T08:02:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
45542025-11-13T08:02:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
45552025-11-13T08:02:02.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45562025-11-13T08:02:02.763ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45572025-11-13T08:02:02.763ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 145.366737ms iteration = 3 status = {"nsuccess":5}
45582025-11-13T08:02:02.764ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45592025-11-13T08:02:02.764ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
45602025-11-13T08:02:02.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
45612025-11-13T08:02:02.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
45622025-11-13T08:02:02.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45632025-11-13T08:02:02.765ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
45642025-11-13T08:02:02.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
45652025-11-13T08:02:02.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
45662025-11-13T08:02:02.765ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
45672025-11-13T08:02:02.766ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 147.338786ms iteration = 37 status = {"config_updated":false}
45682025-11-13T08:02:02.767ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
45692025-11-13T08:02:02.767ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
45702025-11-13T08:02:02.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
45712025-11-13T08:02:02.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
45722025-11-13T08:02:02.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45732025-11-13T08:02:02.768ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
45742025-11-13T08:02:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
45752025-11-13T08:02:02.768ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45762025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
45772025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
45782025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
45792025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45802025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
45812025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
45822025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
45832025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
45842025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
45852025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
45862025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
45872025-11-13T08:02:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
45882025-11-13T08:02:02.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45892025-11-13T08:02:02.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
45902025-11-13T08:02:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
45912025-11-13T08:02:02.771ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 123.366926ms iteration = 4 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
45922025-11-13T08:02:02.772ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
45932025-11-13T08:02:02.772ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
45942025-11-13T08:02:02.772ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
45952025-11-13T08:02:02.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
45962025-11-13T08:02:02.772ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 130.536077ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
45972025-11-13T08:02:02.772ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
45982025-11-13T08:02:02.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
45992025-11-13T08:02:02.773ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46002025-11-13T08:02:02.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
46012025-11-13T08:02:02.774ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6131, tv_nsec: 126730849 } })
46022025-11-13T08:02:02.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46032025-11-13T08:02:02.774ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
46042025-11-13T08:02:02.775ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46052025-11-13T08:02:02.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
46062025-11-13T08:02:02.775ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
46072025-11-13T08:02:02.775ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46082025-11-13T08:02:02.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
46092025-11-13T08:02:02.778ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
46102025-11-13T08:02:02.778ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
46112025-11-13T08:02:02.778ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 136.198718ms iteration = 4 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
46122025-11-13T08:02:02.778ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
46132025-11-13T08:02:02.778ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
46142025-11-13T08:02:02.778ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
46152025-11-13T08:02:02.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
46162025-11-13T08:02:02.779ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
46172025-11-13T08:02:02.779ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 131.602857ms iteration = 13 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
46182025-11-13T08:02:02.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
46192025-11-13T08:02:02.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
46202025-11-13T08:02:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
46212025-11-13T08:02:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
46222025-11-13T08:02:02.782ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
46232025-11-13T08:02:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
46242025-11-13T08:02:02.790ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
46252025-11-13T08:02:02.790ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
46262025-11-13T08:02:02.790ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
46272025-11-13T08:02:02.790ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 182.637531ms iteration = 4 status = {"generation":2}
46282025-11-13T08:02:02.796ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 177.467829ms iteration = 12 status = {"physical_disks_added":0}
46292025-11-13T08:02:02.797ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
46302025-11-13T08:02:02.799ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
46312025-11-13T08:02:02.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
46322025-11-13T08:02:02.799ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 151.06775ms iteration = 7 status = {"errors":[],"requests_created_ok":[]}
46332025-11-13T08:02:02.805ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 157.6506ms iteration = 7 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
46342025-11-13T08:02:02.822ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
46352025-11-13T08:02:02.822ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
46362025-11-13T08:02:02.822ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
46372025-11-13T08:02:02.822ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46382025-11-13T08:02:02.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46392025-11-13T08:02:02.825ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46402025-11-13T08:02:02.829ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46412025-11-13T08:02:02.829ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
46422025-11-13T08:02:02.829ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
46432025-11-13T08:02:02.830ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
46442025-11-13T08:02:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 192.363522ms iteration = 19 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
46452025-11-13T08:02:02.834ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 186.895334ms iteration = 13 status = "NoSitrep"
46462025-11-13T08:02:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
46472025-11-13T08:02:02.835ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
46482025-11-13T08:02:02.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
46492025-11-13T08:02:02.836ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
46502025-11-13T08:02:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 217.726781ms iteration = 19 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
46512025-11-13T08:02:02.841ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
46522025-11-13T08:02:02.841ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
46532025-11-13T08:02:02.841ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
46542025-11-13T08:02:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
46552025-11-13T08:02:02.847ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46562025-11-13T08:02:02.851ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
46572025-11-13T08:02:02.851ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
46582025-11-13T08:02:02.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
46592025-11-13T08:02:02.851ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
46602025-11-13T08:02:02.852ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
46612025-11-13T08:02:02.852ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "dff911df-d1cc-4768-8f94-8f16e1f09995", "content-length": "78", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} })
46622025-11-13T08:02:02.852ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
46632025-11-13T08:02:02.852ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
46642025-11-13T08:02:02.852ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
46652025-11-13T08:02:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
46662025-11-13T08:02:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
46672025-11-13T08:02:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
46682025-11-13T08:02:02.855ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
46692025-11-13T08:02:02.855ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
46702025-11-13T08:02:02.855ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 236.742574ms iteration = 10 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
46712025-11-13T08:02:02.855ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
46722025-11-13T08:02:02.858ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
46732025-11-13T08:02:02.858ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
46742025-11-13T08:02:02.858ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 216.924685ms iteration = 7 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
46752025-11-13T08:02:02.859ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
46762025-11-13T08:02:02.859ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
46772025-11-13T08:02:02.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
46782025-11-13T08:02:02.860ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
46792025-11-13T08:02:02.860ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
46802025-11-13T08:02:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
46812025-11-13T08:02:02.861ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
46822025-11-13T08:02:02.861ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
46832025-11-13T08:02:02.861ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
46842025-11-13T08:02:02.861ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
46852025-11-13T08:02:02.861ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
46862025-11-13T08:02:02.862ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
46872025-11-13T08:02:02.862ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8b9166b-cc2f-4ac6-b4ed-c30b267b92ff", "content-length": "81", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} })
46882025-11-13T08:02:02.862ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e1a6d5a-792c-44a3-9b1f-f7285e3c68b6", "content-length": "26", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} })
46892025-11-13T08:02:02.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
46902025-11-13T08:02:02.862ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
46912025-11-13T08:02:02.863ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
46922025-11-13T08:02:02.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
46932025-11-13T08:02:02.865ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
46942025-11-13T08:02:02.867ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 219.21186ms iteration = 7 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
46952025-11-13T08:02:02.870ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
46962025-11-13T08:02:02.872ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 264.864957ms iteration = 7 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
46972025-11-13T08:02:02.893ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
46982025-11-13T08:02:02.893ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 286.38958ms iteration = 4 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
46992025-11-13T08:02:02.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47002025-11-13T08:02:02.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47012025-11-13T08:02:02.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47022025-11-13T08:02:02.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
47032025-11-13T08:02:02.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47042025-11-13T08:02:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47052025-11-13T08:02:02.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47062025-11-13T08:02:02.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
47072025-11-13T08:02:02.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47082025-11-13T08:02:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47092025-11-13T08:02:02.914ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
47102025-11-13T08:02:02.914ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
47112025-11-13T08:02:02.914ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47122025-11-13T08:02:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47132025-11-13T08:02:02.916ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47142025-11-13T08:02:02.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47152025-11-13T08:02:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47162025-11-13T08:02:02.918ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = DeployNexusRecords step id = Ensure
47172025-11-13T08:02:02.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47182025-11-13T08:02:02.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47192025-11-13T08:02:02.919ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
47202025-11-13T08:02:02.919ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
47212025-11-13T08:02:02.919ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 278.016956ms iteration = 7 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
47222025-11-13T08:02:02.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47232025-11-13T08:02:02.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47242025-11-13T08:02:02.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47252025-11-13T08:02:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
47262025-11-13T08:02:02.922ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
47272025-11-13T08:02:02.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47282025-11-13T08:02:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
47292025-11-13T08:02:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
47302025-11-13T08:02:02.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47312025-11-13T08:02:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47322025-11-13T08:02:02.927ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
47332025-11-13T08:02:02.927ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
47342025-11-13T08:02:02.927ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
47352025-11-13T08:02:02.927ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 320.46696ms iteration = 4 status = {"generation":1}
47362025-11-13T08:02:02.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
47372025-11-13T08:02:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
47382025-11-13T08:02:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
47392025-11-13T08:02:02.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47402025-11-13T08:02:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47412025-11-13T08:02:02.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47422025-11-13T08:02:02.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47432025-11-13T08:02:02.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47442025-11-13T08:02:02.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
47452025-11-13T08:02:02.938ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
47462025-11-13T08:02:02.938ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
47472025-11-13T08:02:02.939ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e31a21d-42a3-4bb1-8f9f-80217b1d3758", "content-length": "81", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} })
47482025-11-13T08:02:02.942ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47492025-11-13T08:02:02.942ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
47502025-11-13T08:02:02.942ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
47512025-11-13T08:02:02.942ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e8a97f5-d24a-4c0d-aea4-f31a0d7379f8", "content-length": "81", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} })
47522025-11-13T08:02:02.945ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 297.242054ms iteration = 7 status = {"disabled":false,"errors":[],"sps":[]}
47532025-11-13T08:02:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
47542025-11-13T08:02:02.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47552025-11-13T08:02:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47562025-11-13T08:02:02.948ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
47572025-11-13T08:02:02.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47582025-11-13T08:02:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
47592025-11-13T08:02:02.953ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
47602025-11-13T08:02:02.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47612025-11-13T08:02:02.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47622025-11-13T08:02:02.955ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
47632025-11-13T08:02:02.955ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
47642025-11-13T08:02:02.955ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "72b08170-7e5b-447c-a345-51253991b890", "content-length": "26", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} })
47652025-11-13T08:02:02.955ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
47662025-11-13T08:02:02.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47672025-11-13T08:02:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47682025-11-13T08:02:02.957ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
47692025-11-13T08:02:02.957ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
47702025-11-13T08:02:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
47712025-11-13T08:02:02.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47722025-11-13T08:02:02.959ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47732025-11-13T08:02:02.960ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47742025-11-13T08:02:02.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
47752025-11-13T08:02:02.963ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
47762025-11-13T08:02:02.963ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
47772025-11-13T08:02:02.964ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
47782025-11-13T08:02:02.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47792025-11-13T08:02:02.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
47802025-11-13T08:02:02.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47812025-11-13T08:02:02.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47822025-11-13T08:02:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47832025-11-13T08:02:02.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47842025-11-13T08:02:02.971ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
47852025-11-13T08:02:02.971ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3f491220-4d4a-43f7-a6db-d964bbcb4f89", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47862025-11-13T08:02:02.971ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47872025-11-13T08:02:02.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
47882025-11-13T08:02:02.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47892025-11-13T08:02:02.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47902025-11-13T08:02:02.973ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "805938d1-8107-443b-9102-127600d0a7ff", "date": "Thu, 13 Nov 2025 08:02:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47912025-11-13T08:02:02.973ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47922025-11-13T08:02:02.973ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
47932025-11-13T08:02:02.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
47942025-11-13T08:02:02.975ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47952025-11-13T08:02:02.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
47962025-11-13T08:02:02.975ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
47972025-11-13T08:02:02.975ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 368.483304ms iteration = 6 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
47982025-11-13T08:02:02.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
47992025-11-13T08:02:02.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48002025-11-13T08:02:02.978ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48012025-11-13T08:02:02.978ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48022025-11-13T08:02:02.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
48032025-11-13T08:02:02.981ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48042025-11-13T08:02:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
48052025-11-13T08:02:02.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48062025-11-13T08:02:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48072025-11-13T08:02:02.984ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
48082025-11-13T08:02:02.985ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
48092025-11-13T08:02:02.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
48102025-11-13T08:02:03.003ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = SledAgent step id = Ensure
48112025-11-13T08:02:03.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48122025-11-13T08:02:03.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48132025-11-13T08:02:03.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48142025-11-13T08:02:03.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48152025-11-13T08:02:03.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48162025-11-13T08:02:03.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
48172025-11-13T08:02:03.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48182025-11-13T08:02:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48192025-11-13T08:02:03.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48202025-11-13T08:02:03.032ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
48212025-11-13T08:02:03.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48222025-11-13T08:02:03.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48232025-11-13T08:02:03.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48242025-11-13T08:02:03.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48252025-11-13T08:02:03.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48262025-11-13T08:02:03.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48272025-11-13T08:02:03.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48282025-11-13T08:02:03.045ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
48292025-11-13T08:02:03.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48302025-11-13T08:02:03.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48312025-11-13T08:02:03.045ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 438.48003ms iteration = 7 status = {}
48322025-11-13T08:02:03.047ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48332025-11-13T08:02:03.052ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
48342025-11-13T08:02:03.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48352025-11-13T08:02:03.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48362025-11-13T08:02:03.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48372025-11-13T08:02:03.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48382025-11-13T08:02:03.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48392025-11-13T08:02:03.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48402025-11-13T08:02:03.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48412025-11-13T08:02:03.063ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
48422025-11-13T08:02:03.108ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
48432025-11-13T08:02:03.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48442025-11-13T08:02:03.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48452025-11-13T08:02:03.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48462025-11-13T08:02:03.118ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
48472025-11-13T08:02:03.118ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 476.183712ms iteration = 4 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
48482025-11-13T08:02:03.118ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48492025-11-13T08:02:03.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48502025-11-13T08:02:03.123ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
48512025-11-13T08:02:03.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48522025-11-13T08:02:03.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48532025-11-13T08:02:03.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48542025-11-13T08:02:03.127ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48552025-11-13T08:02:03.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48562025-11-13T08:02:03.133ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
48572025-11-13T08:02:03.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48582025-11-13T08:02:03.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48592025-11-13T08:02:03.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48602025-11-13T08:02:03.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48612025-11-13T08:02:03.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48622025-11-13T08:02:03.140ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
48632025-11-13T08:02:03.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48642025-11-13T08:02:03.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48652025-11-13T08:02:03.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48662025-11-13T08:02:03.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48672025-11-13T08:02:03.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48682025-11-13T08:02:03.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
48692025-11-13T08:02:03.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48702025-11-13T08:02:03.147ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48712025-11-13T08:02:03.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48722025-11-13T08:02:03.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48732025-11-13T08:02:03.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48742025-11-13T08:02:03.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
48752025-11-13T08:02:03.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48762025-11-13T08:02:03.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48772025-11-13T08:02:03.176ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
48782025-11-13T08:02:03.176ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
48792025-11-13T08:02:03.176ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fdc0675-e336-4311-92ae-b1ec11464fb4", "content-length": "372", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
48802025-11-13T08:02:03.176ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
48812025-11-13T08:02:03.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
48822025-11-13T08:02:03.187ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
48832025-11-13T08:02:03.189ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "945da6c5-fa1e-4b6d-bee2-e429cf6c4146", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
48842025-11-13T08:02:03.192ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = FirewallRules step id = Ensure
48852025-11-13T08:02:03.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48862025-11-13T08:02:03.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48872025-11-13T08:02:03.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48882025-11-13T08:02:03.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48892025-11-13T08:02:03.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48902025-11-13T08:02:03.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48912025-11-13T08:02:03.200ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
48922025-11-13T08:02:03.200ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
48932025-11-13T08:02:03.200ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = efe4cf4f-ec7e-44e6-9118-171ebbba1532 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
48942025-11-13T08:02:03.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
48952025-11-13T08:02:03.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
48962025-11-13T08:02:03.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
48972025-11-13T08:02:03.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
48982025-11-13T08:02:03.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48992025-11-13T08:02:03.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49002025-11-13T08:02:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49012025-11-13T08:02:03.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49022025-11-13T08:02:03.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49032025-11-13T08:02:03.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
49042025-11-13T08:02:03.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
49052025-11-13T08:02:03.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
49062025-11-13T08:02:03.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49072025-11-13T08:02:03.208ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
49082025-11-13T08:02:03.208ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
49092025-11-13T08:02:03.208ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 590.262424ms iteration = 11 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
49102025-11-13T08:02:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
49112025-11-13T08:02:03.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49122025-11-13T08:02:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49132025-11-13T08:02:03.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49142025-11-13T08:02:03.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49152025-11-13T08:02:03.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49162025-11-13T08:02:03.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49172025-11-13T08:02:03.216ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49182025-11-13T08:02:03.220ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
49192025-11-13T08:02:03.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49202025-11-13T08:02:03.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49212025-11-13T08:02:03.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49222025-11-13T08:02:03.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49232025-11-13T08:02:03.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49242025-11-13T08:02:03.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49252025-11-13T08:02:03.225ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6131, tv_nsec: 578165434 } })
49262025-11-13T08:02:03.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49272025-11-13T08:02:03.227ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49282025-11-13T08:02:03.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49292025-11-13T08:02:03.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49302025-11-13T08:02:03.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49312025-11-13T08:02:03.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49322025-11-13T08:02:03.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49332025-11-13T08:02:03.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49342025-11-13T08:02:03.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49352025-11-13T08:02:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
49362025-11-13T08:02:03.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49372025-11-13T08:02:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49382025-11-13T08:02:03.239ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
49392025-11-13T08:02:03.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49402025-11-13T08:02:03.239ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
49412025-11-13T08:02:03.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49422025-11-13T08:02:03.239ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7665bde8-90ba-4ba7-9d92-26726937f87e", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
49432025-11-13T08:02:03.239ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49442025-11-13T08:02:03.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49452025-11-13T08:02:03.240ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
49462025-11-13T08:02:03.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49472025-11-13T08:02:03.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49482025-11-13T08:02:03.241ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "59d1205f-9603-49d1-9094-4260504e22e5", "content-length": "691", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
49492025-11-13T08:02:03.241ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
49502025-11-13T08:02:03.241ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
49512025-11-13T08:02:03.241ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
49522025-11-13T08:02:03.242ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 600.343253ms iteration = 7 status = {}
49532025-11-13T08:02:03.242ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49542025-11-13T08:02:03.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49552025-11-13T08:02:03.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49562025-11-13T08:02:03.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49572025-11-13T08:02:03.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49582025-11-13T08:02:03.245ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49592025-11-13T08:02:03.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49602025-11-13T08:02:03.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49612025-11-13T08:02:03.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49622025-11-13T08:02:03.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49632025-11-13T08:02:03.249ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
49642025-11-13T08:02:03.249ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
49652025-11-13T08:02:03.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49662025-11-13T08:02:03.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49672025-11-13T08:02:03.249ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9032c9e9-64d6-4fd1-9de7-23e0029f8742", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
49682025-11-13T08:02:03.250ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
49692025-11-13T08:02:03.250ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "b6fd2f80-a36c-4429-92ee-8aaa56e1151b", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
49702025-11-13T08:02:03.250ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
49712025-11-13T08:02:03.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49722025-11-13T08:02:03.252ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 610.802408ms iteration = 10 status = {}
49732025-11-13T08:02:03.253ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
49742025-11-13T08:02:03.253ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 604.383611ms iteration = 4 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
49752025-11-13T08:02:03.253ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 11 reason = Dependency
49762025-11-13T08:02:03.253ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
49772025-11-13T08:02:03.253ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49782025-11-13T08:02:03.253ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49792025-11-13T08:02:03.254ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49802025-11-13T08:02:03.255ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
49812025-11-13T08:02:03.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49822025-11-13T08:02:03.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49832025-11-13T08:02:03.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49842025-11-13T08:02:03.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49852025-11-13T08:02:03.258ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49862025-11-13T08:02:03.258ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49872025-11-13T08:02:03.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49882025-11-13T08:02:03.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49892025-11-13T08:02:03.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49902025-11-13T08:02:03.259ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49912025-11-13T08:02:03.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49922025-11-13T08:02:03.263ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
49932025-11-13T08:02:03.263ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
49942025-11-13T08:02:03.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
49952025-11-13T08:02:03.265ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
49962025-11-13T08:02:03.265ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
49972025-11-13T08:02:03.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
49982025-11-13T08:02:03.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
49992025-11-13T08:02:03.266ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "40c84eba-e7bf-4373-b2df-75a18dfd2390", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
50002025-11-13T08:02:03.266ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
50012025-11-13T08:02:03.268ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "bbcc2596-5865-4a75-80c3-08a31681a6cf", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
50022025-11-13T08:02:03.268ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
50032025-11-13T08:02:03.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50042025-11-13T08:02:03.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50052025-11-13T08:02:03.273ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "55530c0e-3cff-472e-833e-90048a78cb9b", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
50062025-11-13T08:02:03.273ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
50072025-11-13T08:02:03.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50082025-11-13T08:02:03.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50092025-11-13T08:02:03.275ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
50102025-11-13T08:02:03.275ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 668.020513ms iteration = 6 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
50112025-11-13T08:02:03.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50122025-11-13T08:02:03.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50132025-11-13T08:02:03.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50142025-11-13T08:02:03.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50152025-11-13T08:02:03.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50162025-11-13T08:02:03.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50172025-11-13T08:02:03.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50182025-11-13T08:02:03.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50192025-11-13T08:02:03.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50202025-11-13T08:02:03.285ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50212025-11-13T08:02:03.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50222025-11-13T08:02:03.292ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
50232025-11-13T08:02:03.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50242025-11-13T08:02:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50252025-11-13T08:02:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50262025-11-13T08:02:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50272025-11-13T08:02:03.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50282025-11-13T08:02:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50292025-11-13T08:02:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50302025-11-13T08:02:03.302ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
50312025-11-13T08:02:03.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50322025-11-13T08:02:03.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50332025-11-13T08:02:03.309ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
50342025-11-13T08:02:03.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50352025-11-13T08:02:03.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50362025-11-13T08:02:03.311ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50372025-11-13T08:02:03.313ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
50382025-11-13T08:02:03.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50392025-11-13T08:02:03.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50402025-11-13T08:02:03.315ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
50412025-11-13T08:02:03.315ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
50422025-11-13T08:02:03.315ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
50432025-11-13T08:02:03.326ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50442025-11-13T08:02:03.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50452025-11-13T08:02:03.328ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
50462025-11-13T08:02:03.328ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50472025-11-13T08:02:03.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50482025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
50492025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
50502025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
50512025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
50522025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50532025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50542025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50552025-11-13T08:02:03.344ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50562025-11-13T08:02:03.345ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
50572025-11-13T08:02:03.345ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
50582025-11-13T08:02:03.345ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
50592025-11-13T08:02:03.345ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
50602025-11-13T08:02:03.345ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
50612025-11-13T08:02:03.348ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
50622025-11-13T08:02:03.348ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
50632025-11-13T08:02:03.377ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
50642025-11-13T08:02:03.377ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
50652025-11-13T08:02:03.378ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
50662025-11-13T08:02:03.378ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
50672025-11-13T08:02:03.381ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ff362d-6eba-43a8-bfa5-542144e38e58", "content-length": "372", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
50682025-11-13T08:02:03.381ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50692025-11-13T08:02:03.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50702025-11-13T08:02:03.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50712025-11-13T08:02:03.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50722025-11-13T08:02:03.414ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50732025-11-13T08:02:03.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50742025-11-13T08:02:03.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50752025-11-13T08:02:03.422ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
50762025-11-13T08:02:03.422ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
50772025-11-13T08:02:03.425ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "05a445b9-aea4-49b4-af1d-ca68c8cdd3e4", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
50782025-11-13T08:02:03.427ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
50792025-11-13T08:02:03.427ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = Dns step id = Ensure
50802025-11-13T08:02:03.427ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
50812025-11-13T08:02:03.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50822025-11-13T08:02:03.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50832025-11-13T08:02:03.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50842025-11-13T08:02:03.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50852025-11-13T08:02:03.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50862025-11-13T08:02:03.434ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
50872025-11-13T08:02:03.435ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
50882025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
50892025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
50902025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
50912025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
50922025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
50932025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3461060a-5a8f-4f4c-834b-79d653598fda", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
50942025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50952025-11-13T08:02:03.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
50962025-11-13T08:02:03.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
50972025-11-13T08:02:03.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50982025-11-13T08:02:03.437ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
50992025-11-13T08:02:03.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51002025-11-13T08:02:03.440ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
51012025-11-13T08:02:03.441ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
51022025-11-13T08:02:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
51032025-11-13T08:02:03.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51042025-11-13T08:02:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51052025-11-13T08:02:03.444ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
51062025-11-13T08:02:03.444ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
51072025-11-13T08:02:03.444ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
51082025-11-13T08:02:03.444ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
51092025-11-13T08:02:03.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51102025-11-13T08:02:03.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51112025-11-13T08:02:03.445ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "8c8c59ec-81d7-4288-93c1-55eefef448f6", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
51122025-11-13T08:02:03.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51132025-11-13T08:02:03.446ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 193.234659ms iteration = 11 status = {}
51142025-11-13T08:02:03.449ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
51152025-11-13T08:02:03.449ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51162025-11-13T08:02:03.449ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51172025-11-13T08:02:03.452ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
51182025-11-13T08:02:03.452ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
51192025-11-13T08:02:03.452ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
51202025-11-13T08:02:03.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51212025-11-13T08:02:03.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51222025-11-13T08:02:03.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51232025-11-13T08:02:03.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51242025-11-13T08:02:03.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51252025-11-13T08:02:03.458ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
51262025-11-13T08:02:03.472ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "8cfb66d7-8fb2-4d1f-b024-3f1e016b3734", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
51272025-11-13T08:02:03.472ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
51282025-11-13T08:02:03.472ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
51292025-11-13T08:02:03.479ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "43618306-4b04-4014-a17d-ac4f118e2cd5", "content-length": "682", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
51302025-11-13T08:02:03.479ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51312025-11-13T08:02:03.479ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51322025-11-13T08:02:03.483ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51332025-11-13T08:02:03.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51342025-11-13T08:02:03.487ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51352025-11-13T08:02:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51362025-11-13T08:02:03.489ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
51372025-11-13T08:02:03.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
51382025-11-13T08:02:03.496ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 854.50824ms iteration = 7 status = {}
51392025-11-13T08:02:03.501ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
51402025-11-13T08:02:03.501ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
51412025-11-13T08:02:03.501ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
51422025-11-13T08:02:03.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51432025-11-13T08:02:03.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51442025-11-13T08:02:03.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51452025-11-13T08:02:03.507ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
51462025-11-13T08:02:03.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51472025-11-13T08:02:03.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51482025-11-13T08:02:03.508ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
51492025-11-13T08:02:03.508ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
51502025-11-13T08:02:03.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51512025-11-13T08:02:03.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51522025-11-13T08:02:03.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51532025-11-13T08:02:03.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
51542025-11-13T08:02:03.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51552025-11-13T08:02:03.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51562025-11-13T08:02:03.515ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
51572025-11-13T08:02:03.515ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
51582025-11-13T08:02:03.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51592025-11-13T08:02:03.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51602025-11-13T08:02:03.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51612025-11-13T08:02:03.521ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
51622025-11-13T08:02:03.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51632025-11-13T08:02:03.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51642025-11-13T08:02:03.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51652025-11-13T08:02:03.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51662025-11-13T08:02:03.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51672025-11-13T08:02:03.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51682025-11-13T08:02:03.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51692025-11-13T08:02:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
51702025-11-13T08:02:03.531ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
51712025-11-13T08:02:03.531ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
51722025-11-13T08:02:03.531ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = OmicronZones step id = Cleanup
51732025-11-13T08:02:03.531ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = OmicronZones step id = Cleanup
51742025-11-13T08:02:03.531ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = PhysicalDisks step id = Cleanup
51752025-11-13T08:02:03.531ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = Clickhouse step id = Ensure
51762025-11-13T08:02:03.531ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = Clickhouse step id = Ensure
51772025-11-13T08:02:03.561ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
51782025-11-13T08:02:03.566ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "38788a6d-80d8-4c5f-a22c-90baf1ce9b67", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
51792025-11-13T08:02:03.567ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = SupportBundles step id = Cleanup
51802025-11-13T08:02:03.567ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51812025-11-13T08:02:03.567ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51822025-11-13T08:02:03.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51832025-11-13T08:02:03.570ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
51842025-11-13T08:02:03.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51852025-11-13T08:02:03.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51862025-11-13T08:02:03.572ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51872025-11-13T08:02:03.572ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51882025-11-13T08:02:03.573ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51892025-11-13T08:02:03.576ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
51902025-11-13T08:02:03.614ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = OmicronZones step id = Cleanup
51912025-11-13T08:02:03.614ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
51922025-11-13T08:02:03.614ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
51932025-11-13T08:02:03.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51942025-11-13T08:02:03.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51952025-11-13T08:02:03.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51962025-11-13T08:02:03.617ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
51972025-11-13T08:02:03.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
51982025-11-13T08:02:03.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
51992025-11-13T08:02:03.654ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
52002025-11-13T08:02:03.654ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
52012025-11-13T08:02:03.654ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = Cockroach step id = Ensure
52022025-11-13T08:02:03.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
52032025-11-13T08:02:03.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
52042025-11-13T08:02:03.668ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
52052025-11-13T08:02:03.668ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = f1d9ffe7-967a-44dd-b445-d568b015dad0 step component = MgsUpdates step id = Ensure
52062025-11-13T08:02:03.668ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 11 reason = Dependency
52072025-11-13T08:02:03.668ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 5 reason = Dependency
52082025-11-13T08:02:03.668ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52092025-11-13T08:02:03.668ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52102025-11-13T08:02:03.668ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.060944458s iteration = 5 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4980,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":103659688,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":103659628,"secs":0}},"event_index":1,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":103687938,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":207079901,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":207079841,"secs":0}},"event_index":2,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":310817550,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":84776367,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":84776307,"secs":0}},"event_index":3,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":395655928,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":188339153,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":188339103,"secs":0}},"event_index":4,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":584016151,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":235287655,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":235287615,"secs":0}},"event_index":5,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":819320347,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":104014125,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":104014065,"secs":0}},"event_index":6,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":923342252,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4451,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4400,"secs":0}},"event_index":7,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":923357933,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2840,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2789,"secs":0}},"event_index":8,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":923363013,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3430,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3380,"secs":0}},"event_index":9,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":923368413,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2120,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2070,"secs":0}},"event_index":10,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":923372493,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35819884,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":35819844,"secs":0}},"event_index":11,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":959193957,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":46861212,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":46861181,"secs":0}},"event_index":12,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":6063128,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":40184260,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":40184210,"secs":0}},"event_index":13,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":46266189,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":14169919,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":14169879,"secs":0}},"event_index":14,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":60451688,"secs":1}},{"data":{"attempt_elapsed":{"nanos":9630,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":9590,"secs":0}},"event_index":15,"execution_id":"8f6213b0-0953-43c5-b134-e3061fff7d57","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":60471098,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
52112025-11-13T08:02:03.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
52122025-11-13T08:02:03.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52132025-11-13T08:02:03.670ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52142025-11-13T08:02:03.670ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52152025-11-13T08:02:03.673ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
52162025-11-13T08:02:03.673ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52172025-11-13T08:02:03.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52182025-11-13T08:02:03.673ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52192025-11-13T08:02:03.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
52202025-11-13T08:02:03.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52212025-11-13T08:02:03.720ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52222025-11-13T08:02:03.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52232025-11-13T08:02:03.722ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52242025-11-13T08:02:03.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
52252025-11-13T08:02:03.724ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
52262025-11-13T08:02:03.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
52272025-11-13T08:02:03.726ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
52282025-11-13T08:02:03.726ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 58.048711ms iteration = 11 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
52292025-11-13T08:02:03.728ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(07399175-748d-4999-b636-5f12bc3d81dd, 0), (6ebc3749-17f1-41eb-93de-ab321c41fd59, 0), (e8e23df8-464c-42da-a8a1-dd6852e90c3c, 0), (f774ec9f-33f4-4ebc-a4df-0f8c6cbebd00, 0)] collection_id = 07399175-748d-4999-b636-5f12bc3d81dd
52302025-11-13T08:02:03.728ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52312025-11-13T08:02:03.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
52322025-11-13T08:02:03.930ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 07399175-748d-4999-b636-5f12bc3d81dd file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
52332025-11-13T08:02:03.930ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52342025-11-13T08:02:03.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
52352025-11-13T08:02:03.942ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(e8e23df8-464c-42da-a8a1-dd6852e90c3c, 0), (f774ec9f-33f4-4ebc-a4df-0f8c6cbebd00, 0), (55b45047-b092-4620-af64-6b27b076a7c5, 0)]
52362025-11-13T08:02:03.942ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6132, tv_nsec: 294786393 } })
52372025-11-13T08:02:03.974ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6132, tv_nsec: 326896944 } })
52382025-11-13T08:02:03.975ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
52392025-11-13T08:02:03.975ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
52402025-11-13T08:02:03.975ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
52412025-11-13T08:02:03.975ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3331ec0-ba31-43b2-a196-1dba5addec1d", "content-length": "476", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52422025-11-13T08:02:03.975ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
52432025-11-13T08:02:03.976ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "57e3e812-3369-4fc8-a4b7-e341afc43619", "content-length": "734", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52442025-11-13T08:02:03.976ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
52452025-11-13T08:02:03.976ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "57b31f68-7f97-4b73-b5a4-bd3a9e99bfe8", "content-length": "111", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52462025-11-13T08:02:03.976ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
52472025-11-13T08:02:03.977ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c7a2251-a0af-4cbc-bd31-e391e560836f", "content-length": "111", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52482025-11-13T08:02:03.977ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
52492025-11-13T08:02:03.977ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "762f37e6-2b97-449a-8c43-70af1ddd1421", "content-length": "163", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52502025-11-13T08:02:03.977ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
52512025-11-13T08:02:03.978ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c6ca571-5047-459e-9439-2e85a1d8ea05", "content-length": "163", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52522025-11-13T08:02:03.978ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
52532025-11-13T08:02:03.978ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35ac412c-6f0f-42f6-acda-fc5a0b72af94", "content-length": "166", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52542025-11-13T08:02:03.978ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
52552025-11-13T08:02:03.978ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a40e337-cc30-4b4c-98b5-c18a31926613", "content-length": "166", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52562025-11-13T08:02:03.978ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
52572025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc771d2f-ffb1-4783-bbb8-21a192cb1547", "content-length": "702", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52582025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
52592025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f21c9a73-7090-4b1d-8922-48b0f1a5fcdb", "content-length": "727", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52602025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
52612025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "22bfcd2d-6405-4a08-ae6e-50d0ccd7aaf5", "content-length": "729", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52622025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
52632025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5221b83-56c2-43bc-9b00-2d946f296666", "content-length": "728", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52642025-11-13T08:02:03.979ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
52652025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "94534987-9de4-451f-933a-1b6feab3d9cd", "content-length": "722", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52662025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
52672025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cdc9dd3-69df-44b9-8fd9-b3275dbea2b1", "content-length": "10", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52682025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
52692025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70d18d79-6d95-45e4-a809-c452cb7a87a1", "content-length": "144", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52702025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
52712025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab15f497-0fd3-431f-ac62-e5adadac2a95", "content-length": "147", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52722025-11-13T08:02:03.980ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
52732025-11-13T08:02:03.981ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cb528a5-60c2-4eb6-8620-ce63b59f587b", "content-length": "109", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52742025-11-13T08:02:03.981ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
52752025-11-13T08:02:03.981ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6932fa2-6844-4044-8b7e-c2c07e1bcede", "content-length": "109", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52762025-11-13T08:02:03.981ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
52772025-11-13T08:02:03.982ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd21a52c-8a26-47b9-ac0d-120fbdd1e141", "content-length": "163", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52782025-11-13T08:02:03.982ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
52792025-11-13T08:02:03.982ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8124a0da-d1d6-44ed-8263-d897ede603d9", "content-length": "163", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52802025-11-13T08:02:03.982ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
52812025-11-13T08:02:03.983ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d024c75-d936-4af6-b6b7-051c4c4d1973", "content-length": "166", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52822025-11-13T08:02:03.983ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
52832025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fdd9b48-16b3-464b-bd03-e5cb071461b6", "content-length": "166", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52842025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
52852025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b32dc65-0426-4b85-80b4-d5d8f08353f1", "content-length": "702", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52862025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
52872025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "94755bfc-e883-4e10-9e00-642b6dbbd8bd", "content-length": "727", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52882025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
52892025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6eecf1b-9239-4ab5-b46b-0eb0fa176714", "content-length": "729", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52902025-11-13T08:02:03.984ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
52912025-11-13T08:02:03.985ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3649bb2-a428-4769-81e3-d737dca1ad59", "content-length": "728", "date": "Thu, 13 Nov 2025 08:02:03 GMT"} })
52922025-11-13T08:02:03.985ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52932025-11-13T08:02:03.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
52942025-11-13T08:02:03.987ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52952025-11-13T08:02:03.990ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
52962025-11-13T08:02:03.990ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52972025-11-13T08:02:03.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
52982025-11-13T08:02:03.992ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52992025-11-13T08:02:03.994ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
53002025-11-13T08:02:03.994ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53012025-11-13T08:02:03.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53022025-11-13T08:02:04.028ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
53032025-11-13T08:02:04.028ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
53042025-11-13T08:02:04.123ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f6f8b42-0739-4ce9-bc53-af0602e078e6", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53052025-11-13T08:02:04.125ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
53062025-11-13T08:02:04.125ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
53072025-11-13T08:02:04.125ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
53082025-11-13T08:02:04.125ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
53092025-11-13T08:02:04.125ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
53102025-11-13T08:02:04.125ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
53112025-11-13T08:02:04.125ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
53122025-11-13T08:02:04.125ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
53132025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b319acaf-7154-49c1-a4e5-a94e39542501", "content-length": "64", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53142025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "031eb4f3-7c6c-43a9-9214-4f61865bd486", "content-length": "64", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53152025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
53162025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
53172025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd046843-e810-4d93-a3a0-3b00c92d4631", "content-length": "64", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53182025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
53192025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "07f684e6-05bc-40d2-80c9-036ab5ae2832", "content-length": "64", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53202025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
53212025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8e5591c-52fe-4b05-8aaf-59fdd5d8d347", "content-length": "64", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53222025-11-13T08:02:04.126ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
53232025-11-13T08:02:04.163ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "65257899-6701-4875-9fa9-3da778e9d119", "content-length": "393095", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53242025-11-13T08:02:04.165ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf1fe1a0-a126-4638-baa3-c7ad22583201", "content-length": "384815", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53252025-11-13T08:02:04.167ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f0ec7b1-a76a-4d03-94cc-6045a8391d22", "content-length": "385514", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53262025-11-13T08:02:04.171ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e991ce53-6732-4ebb-b10c-86a739028dc7", "content-length": "411509", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53272025-11-13T08:02:04.174ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7de5daa8-9b06-4b4c-908c-eefebad23f21", "content-length": "380139", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53282025-11-13T08:02:04.177ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
53292025-11-13T08:02:04.177ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
53302025-11-13T08:02:04.177ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
53312025-11-13T08:02:04.177ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
53322025-11-13T08:02:04.177ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
53332025-11-13T08:02:04.208ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
53342025-11-13T08:02:04.208ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
53352025-11-13T08:02:04.211ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f6c05c9-bc93-4491-a59a-d04fb50b4a3a", "content-length": "120", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53362025-11-13T08:02:04.212ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
53372025-11-13T08:02:04.302ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
53382025-11-13T08:02:04.302ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
53392025-11-13T08:02:04.303ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "3014b95d-b6ff-4ead-b3aa-54d5e47bc94b", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53402025-11-13T08:02:04.303ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
53412025-11-13T08:02:04.304ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
53422025-11-13T08:02:04.304ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
53432025-11-13T08:02:04.305ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "12f5bcbc-3ac6-4893-8d10-36763d3c582f", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53442025-11-13T08:02:04.305ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
53452025-11-13T08:02:04.306ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
53462025-11-13T08:02:04.306ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
53472025-11-13T08:02:04.307ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b38666d7-f4d3-4e6c-82d5-c37d091d9d16", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:02:04 GMT"} })
53482025-11-13T08:02:04.307ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
53492025-11-13T08:02:04.309ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
53502025-11-13T08:02:04.309ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
53512025-11-13T08:02:04.311ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53522025-11-13T08:02:04.311ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53532025-11-13T08:02:04.313ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53542025-11-13T08:02:04.315ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
53552025-11-13T08:02:04.315ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53562025-11-13T08:02:04.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
53572025-11-13T08:02:04.525ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 25faec09-1a95-4211-b5bb-8965f08ec3c6 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
53582025-11-13T08:02:04.526ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 25faec09-1a95-4211-b5bb-8965f08ec3c6 time_started = 2025-11-13 08:02:03.975249 UTC
53592025-11-13T08:02:04.526ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 18 reason = Dependency
53602025-11-13T08:02:04.526ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
53612025-11-13T08:02:04.526ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 857.506428ms iteration = 5 status = {"collection_id":"25faec09-1a95-4211-b5bb-8965f08ec3c6","time_done":"2025-11-13 08:02:04.309362 UTC","time_started":"2025-11-13 08:02:03.975249 UTC"}
53622025-11-13T08:02:04.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
53632025-11-13T08:02:04.528ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53642025-11-13T08:02:04.532ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
53652025-11-13T08:02:04.532ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
53662025-11-13T08:02:04.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
53672025-11-13T08:02:04.550ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
53682025-11-13T08:02:04.550ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
53692025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 13 reason = Dependency
53702025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 70.030837ms iteration = 18 status = {"Loaded":{"collection_id":"036cf7b1-53e4-423a-bec9-1e7a1c588e6a","time_loaded":"2025-11-13T08:02:04.526016040Z","time_started":"2025-11-13T08:02:03.980290Z"}}
53712025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 12 reason = Dependency
53722025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 7 reason = Dependency
53732025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53742025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53752025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
53762025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53772025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53782025-11-13T08:02:04.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
53792025-11-13T08:02:04.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53802025-11-13T08:02:04.598ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53812025-11-13T08:02:04.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53822025-11-13T08:02:04.600ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
53832025-11-13T08:02:04.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53842025-11-13T08:02:04.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53852025-11-13T08:02:04.601ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
53862025-11-13T08:02:04.601ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53872025-11-13T08:02:04.601ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
53882025-11-13T08:02:04.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
53892025-11-13T08:02:04.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53902025-11-13T08:02:04.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
53912025-11-13T08:02:04.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53922025-11-13T08:02:04.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53932025-11-13T08:02:04.603ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
53942025-11-13T08:02:04.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
53952025-11-13T08:02:04.604ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
53962025-11-13T08:02:04.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
53972025-11-13T08:02:04.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
53982025-11-13T08:02:04.605ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53992025-11-13T08:02:04.606ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
54002025-11-13T08:02:04.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54012025-11-13T08:02:04.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54022025-11-13T08:02:04.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54032025-11-13T08:02:04.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54042025-11-13T08:02:04.608ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
54052025-11-13T08:02:04.608ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
54062025-11-13T08:02:04.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
54072025-11-13T08:02:04.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54082025-11-13T08:02:04.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54092025-11-13T08:02:04.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54102025-11-13T08:02:04.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54112025-11-13T08:02:04.610ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
54122025-11-13T08:02:04.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54132025-11-13T08:02:04.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54142025-11-13T08:02:04.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54152025-11-13T08:02:04.613ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
54162025-11-13T08:02:04.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54172025-11-13T08:02:04.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54182025-11-13T08:02:04.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
54192025-11-13T08:02:04.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54202025-11-13T08:02:04.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54212025-11-13T08:02:04.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54222025-11-13T08:02:04.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
54232025-11-13T08:02:04.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54242025-11-13T08:02:04.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54252025-11-13T08:02:04.619ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
54262025-11-13T08:02:04.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54272025-11-13T08:02:04.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54282025-11-13T08:02:04.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54292025-11-13T08:02:04.623ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
54302025-11-13T08:02:04.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54312025-11-13T08:02:04.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54322025-11-13T08:02:04.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54332025-11-13T08:02:04.626ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
54342025-11-13T08:02:04.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
54352025-11-13T08:02:04.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
54362025-11-13T08:02:04.628ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
54372025-11-13T08:02:04.628ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 32.808275ms iteration = 7 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"036cf7b1-53e4-423a-bec9-1e7a1c588e6a","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
54382025-11-13T08:02:04.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54392025-11-13T08:02:04.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54402025-11-13T08:02:04.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54412025-11-13T08:02:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54422025-11-13T08:02:04.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54432025-11-13T08:02:04.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
54442025-11-13T08:02:04.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54452025-11-13T08:02:04.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54462025-11-13T08:02:04.657ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54472025-11-13T08:02:04.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54482025-11-13T08:02:04.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54492025-11-13T08:02:04.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
54502025-11-13T08:02:04.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54512025-11-13T08:02:04.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54522025-11-13T08:02:04.662ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54532025-11-13T08:02:04.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
54542025-11-13T08:02:04.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54552025-11-13T08:02:04.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54562025-11-13T08:02:04.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54572025-11-13T08:02:04.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54582025-11-13T08:02:04.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54592025-11-13T08:02:04.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
54602025-11-13T08:02:04.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54612025-11-13T08:02:04.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54622025-11-13T08:02:04.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54632025-11-13T08:02:04.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54642025-11-13T08:02:04.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54652025-11-13T08:02:04.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
54662025-11-13T08:02:04.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54672025-11-13T08:02:04.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54682025-11-13T08:02:04.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54692025-11-13T08:02:04.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54702025-11-13T08:02:04.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54712025-11-13T08:02:04.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
54722025-11-13T08:02:04.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54732025-11-13T08:02:04.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54742025-11-13T08:02:04.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54752025-11-13T08:02:04.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54762025-11-13T08:02:04.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54772025-11-13T08:02:04.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54782025-11-13T08:02:04.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54792025-11-13T08:02:04.712ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
54802025-11-13T08:02:04.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54812025-11-13T08:02:04.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54822025-11-13T08:02:04.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54832025-11-13T08:02:04.716ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
54842025-11-13T08:02:04.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54852025-11-13T08:02:04.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54862025-11-13T08:02:04.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54872025-11-13T08:02:04.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54882025-11-13T08:02:04.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54892025-11-13T08:02:04.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54902025-11-13T08:02:04.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54912025-11-13T08:02:04.731ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
54922025-11-13T08:02:04.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54932025-11-13T08:02:04.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54942025-11-13T08:02:04.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54952025-11-13T08:02:04.734ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 138.074154ms iteration = 13 status = {"physical_disks_added":0}
54962025-11-13T08:02:04.735ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
54972025-11-13T08:02:04.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
54982025-11-13T08:02:04.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
54992025-11-13T08:02:04.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55002025-11-13T08:02:04.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55012025-11-13T08:02:04.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55022025-11-13T08:02:04.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55032025-11-13T08:02:04.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55042025-11-13T08:02:04.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
55052025-11-13T08:02:04.744ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55062025-11-13T08:02:04.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55072025-11-13T08:02:04.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55082025-11-13T08:02:04.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55092025-11-13T08:02:04.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55102025-11-13T08:02:04.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
55112025-11-13T08:02:04.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55122025-11-13T08:02:04.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55132025-11-13T08:02:04.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55142025-11-13T08:02:04.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55152025-11-13T08:02:04.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55162025-11-13T08:02:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
55172025-11-13T08:02:04.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55182025-11-13T08:02:04.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55192025-11-13T08:02:04.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55202025-11-13T08:02:04.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55212025-11-13T08:02:04.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55222025-11-13T08:02:04.764ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
55232025-11-13T08:02:04.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55242025-11-13T08:02:04.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55252025-11-13T08:02:04.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55262025-11-13T08:02:04.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55272025-11-13T08:02:04.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55282025-11-13T08:02:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
55292025-11-13T08:02:04.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55302025-11-13T08:02:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55312025-11-13T08:02:04.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55322025-11-13T08:02:04.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55332025-11-13T08:02:04.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55342025-11-13T08:02:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
55352025-11-13T08:02:04.779ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55362025-11-13T08:02:04.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55372025-11-13T08:02:04.818ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
55382025-11-13T08:02:04.818ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
55392025-11-13T08:02:04.818ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 30a50609-6aff-4d38-9050-1c7bb3823b60 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
55402025-11-13T08:02:04.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55412025-11-13T08:02:04.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55422025-11-13T08:02:04.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55432025-11-13T08:02:04.823ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
55442025-11-13T08:02:04.823ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
55452025-11-13T08:02:04.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
55462025-11-13T08:02:04.826ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
55472025-11-13T08:02:04.826ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
55482025-11-13T08:02:04.826ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 230.43399ms iteration = 12 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
55492025-11-13T08:02:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 286 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 5437b143-612f-401d-8e3c-bd2a6f4a4c2e response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
55502025-11-13T08:02:07.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 38 reason = Timeout
55512025-11-13T08:02:07.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55522025-11-13T08:02:07.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55532025-11-13T08:02:07.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55542025-11-13T08:02:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
55552025-11-13T08:02:07.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55562025-11-13T08:02:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55572025-11-13T08:02:07.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
55582025-11-13T08:02:07.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.217342ms iteration = 38 status = {"config_updated":false}
55592025-11-13T08:02:12.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 20 reason = Timeout
55602025-11-13T08:02:12.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 39 reason = Timeout
55612025-11-13T08:02:12.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55622025-11-13T08:02:12.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55632025-11-13T08:02:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55642025-11-13T08:02:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55652025-11-13T08:02:12.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55662025-11-13T08:02:12.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55672025-11-13T08:02:12.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
55682025-11-13T08:02:12.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55692025-11-13T08:02:12.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
55702025-11-13T08:02:12.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
55712025-11-13T08:02:12.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55722025-11-13T08:02:12.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
55732025-11-13T08:02:12.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55742025-11-13T08:02:12.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55752025-11-13T08:02:12.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
55762025-11-13T08:02:12.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.866695ms iteration = 39 status = {"config_updated":false}
55772025-11-13T08:02:12.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55782025-11-13T08:02:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
55792025-11-13T08:02:12.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
55802025-11-13T08:02:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
55812025-11-13T08:02:12.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 20 reason = Timeout
55822025-11-13T08:02:12.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55832025-11-13T08:02:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55842025-11-13T08:02:12.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
55852025-11-13T08:02:12.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
55862025-11-13T08:02:12.645ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
55872025-11-13T08:02:12.645ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.845198ms iteration = 20 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
55882025-11-13T08:02:12.687ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.737289ms iteration = 20 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
55892025-11-13T08:02:15.472ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:53638
55902025-11-13T08:02:15.472ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 7deff55c-e560-4219-b3ba-52a00fc57bd6 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
55912025-11-13T08:02:15.472ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 7deff55c-e560-4219-b3ba-52a00fc57bd6 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
55922025-11-13T08:02:15.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 7deff55c-e560-4219-b3ba-52a00fc57bd6 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
55932025-11-13T08:02:15.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2618 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 7deff55c-e560-4219-b3ba-52a00fc57bd6 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
55942025-11-13T08:02:15.475ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 2d194341-e39d-49a0-ab34-bed0b8861e65 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
55952025-11-13T08:02:15.475ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 2d194341-e39d-49a0-ab34-bed0b8861e65 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
55962025-11-13T08:02:15.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 2d194341-e39d-49a0-ab34-bed0b8861e65 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
55972025-11-13T08:02:15.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2209 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 2d194341-e39d-49a0-ab34-bed0b8861e65 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
55982025-11-13T08:02:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 233 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = f73ec2f9-2146-40c4-b2fe-899653cb465c response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
55992025-11-13T08:02:17.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 19 reason = Timeout
56002025-11-13T08:02:17.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 40 reason = Timeout
56012025-11-13T08:02:17.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
56022025-11-13T08:02:17.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56032025-11-13T08:02:17.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
56042025-11-13T08:02:17.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56052025-11-13T08:02:17.620ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56062025-11-13T08:02:17.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56072025-11-13T08:02:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
56082025-11-13T08:02:17.624ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
56092025-11-13T08:02:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
56102025-11-13T08:02:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56112025-11-13T08:02:17.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56122025-11-13T08:02:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56132025-11-13T08:02:17.625ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 036cf7b1-53e4-423a-bec9-1e7a1c588e6a original_time_started = 2025-11-13 08:02:03.980290 UTC
56142025-11-13T08:02:17.625ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.295313ms iteration = 19 status = {"Loaded":{"collection_id":"036cf7b1-53e4-423a-bec9-1e7a1c588e6a","time_loaded":"2025-11-13T08:02:17.618622391Z","time_started":"2025-11-13T08:02:03.980290Z"}}
56152025-11-13T08:02:17.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
56162025-11-13T08:02:17.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.64169ms iteration = 40 status = {"config_updated":false}
56172025-11-13T08:02:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 14 reason = Timeout
56182025-11-13T08:02:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
56192025-11-13T08:02:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 14 reason = Timeout
56202025-11-13T08:02:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
56212025-11-13T08:02:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
56222025-11-13T08:02:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
56232025-11-13T08:02:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
56242025-11-13T08:02:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
56252025-11-13T08:02:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56262025-11-13T08:02:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56272025-11-13T08:02:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
56282025-11-13T08:02:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
56292025-11-13T08:02:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
56302025-11-13T08:02:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
56312025-11-13T08:02:17.653ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.321235ms iteration = 14 status = "NoSitrep"
56322025-11-13T08:02:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
56332025-11-13T08:02:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
56342025-11-13T08:02:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
56352025-11-13T08:02:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
56362025-11-13T08:02:17.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56372025-11-13T08:02:17.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
56382025-11-13T08:02:17.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
56392025-11-13T08:02:17.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.68343ms iteration = 14 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
56402025-11-13T08:02:22.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 21 reason = Timeout
56412025-11-13T08:02:22.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 41 reason = Timeout
56422025-11-13T08:02:22.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56432025-11-13T08:02:22.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56442025-11-13T08:02:22.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56452025-11-13T08:02:22.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56462025-11-13T08:02:22.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56472025-11-13T08:02:22.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56482025-11-13T08:02:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56492025-11-13T08:02:22.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56502025-11-13T08:02:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
56512025-11-13T08:02:22.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56522025-11-13T08:02:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56532025-11-13T08:02:22.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56542025-11-13T08:02:22.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
56552025-11-13T08:02:22.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.198762ms iteration = 41 status = {"config_updated":false}
56562025-11-13T08:02:22.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56572025-11-13T08:02:22.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56582025-11-13T08:02:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56592025-11-13T08:02:22.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
56602025-11-13T08:02:22.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
56612025-11-13T08:02:22.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
56622025-11-13T08:02:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 21 reason = Timeout
56632025-11-13T08:02:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56642025-11-13T08:02:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56652025-11-13T08:02:22.644ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
56662025-11-13T08:02:22.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
56672025-11-13T08:02:22.644ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.95029ms iteration = 21 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
56682025-11-13T08:02:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
56692025-11-13T08:02:22.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.350936ms iteration = 21 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
56702025-11-13T08:02:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 412 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 7d50725c-40d7-489e-b4be-e5a3bfd722a6 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
56712025-11-13T08:02:27.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 42 reason = Timeout
56722025-11-13T08:02:27.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56732025-11-13T08:02:27.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56742025-11-13T08:02:27.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56752025-11-13T08:02:27.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
56762025-11-13T08:02:27.622ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
56772025-11-13T08:02:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
56782025-11-13T08:02:27.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
56792025-11-13T08:02:27.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.958037ms iteration = 42 status = {"config_updated":false}
56802025-11-13T08:02:30.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 25110abc-6629-4db1-ae7b-8ffc9d66b75e uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
56812025-11-13T08:02:30.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 25110abc-6629-4db1-ae7b-8ffc9d66b75e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
56822025-11-13T08:02:30.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 25110abc-6629-4db1-ae7b-8ffc9d66b75e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
56832025-11-13T08:02:30.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3682 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 25110abc-6629-4db1-ae7b-8ffc9d66b75e response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
56842025-11-13T08:02:30.477ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 81e5ad87-97f6-440c-b7f3-2560267343a5 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
56852025-11-13T08:02:30.477ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 81e5ad87-97f6-440c-b7f3-2560267343a5 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
56862025-11-13T08:02:30.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 81e5ad87-97f6-440c-b7f3-2560267343a5 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
56872025-11-13T08:02:30.480ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2877 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 81e5ad87-97f6-440c-b7f3-2560267343a5 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
56882025-11-13T08:02:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 8 reason = Timeout
56892025-11-13T08:02:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 8 reason = Timeout
56902025-11-13T08:02:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
56912025-11-13T08:02:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 8 reason = Timeout
56922025-11-13T08:02:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
56932025-11-13T08:02:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
56942025-11-13T08:02:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
56952025-11-13T08:02:32.611ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6160, tv_nsec: 963815663 } })
56962025-11-13T08:02:32.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6160, tv_nsec: 963884284 } })
56972025-11-13T08:02:32.611ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
56982025-11-13T08:02:32.618ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 14 reason = Timeout
56992025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.72207ms iteration = 8 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
57002025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 20 reason = Timeout
57012025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57022025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 12 reason = Timeout
57032025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
57042025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 22 reason = Timeout
57052025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
57062025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 43 reason = Timeout
57072025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57082025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57092025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
57102025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57112025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
57122025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57132025-11-13T08:02:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57142025-11-13T08:02:32.621ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57152025-11-13T08:02:32.622ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57162025-11-13T08:02:32.623ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57172025-11-13T08:02:32.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57182025-11-13T08:02:32.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57192025-11-13T08:02:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
57202025-11-13T08:02:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
57212025-11-13T08:02:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
57222025-11-13T08:02:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
57232025-11-13T08:02:32.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
57242025-11-13T08:02:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
57252025-11-13T08:02:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
57262025-11-13T08:02:32.628ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57272025-11-13T08:02:32.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57282025-11-13T08:02:32.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
57292025-11-13T08:02:32.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57302025-11-13T08:02:32.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57312025-11-13T08:02:32.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
57322025-11-13T08:02:32.630ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
57332025-11-13T08:02:32.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
57342025-11-13T08:02:32.633ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 036cf7b1-53e4-423a-bec9-1e7a1c588e6a original_time_started = 2025-11-13 08:02:03.980290 UTC
57352025-11-13T08:02:32.633ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 14.366532ms iteration = 20 status = {"Loaded":{"collection_id":"036cf7b1-53e4-423a-bec9-1e7a1c588e6a","time_loaded":"2025-11-13T08:02:32.618969666Z","time_started":"2025-11-13T08:02:03.980290Z"}}
57362025-11-13T08:02:32.639ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
57372025-11-13T08:02:32.639ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 20.415711ms iteration = 43 status = {"config_updated":false}
57382025-11-13T08:02:32.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57392025-11-13T08:02:32.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
57402025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 8 reason = Timeout
57412025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 8 reason = Timeout
57422025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57432025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 8 reason = Timeout
57442025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
57452025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
57462025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 22 reason = Timeout
57472025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
57482025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 12 reason = Timeout
57492025-11-13T08:02:32.643ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
57502025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 8 reason = Timeout
57512025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
57522025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
57532025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 8 reason = Timeout
57542025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
57552025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 8 reason = Timeout
57562025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
57572025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
57582025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
57592025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57602025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57612025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
57622025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
57632025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
57642025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
57652025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
57662025-11-13T08:02:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
57672025-11-13T08:02:32.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
57682025-11-13T08:02:32.647ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
57692025-11-13T08:02:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
57702025-11-13T08:02:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
57712025-11-13T08:02:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 8 reason = Timeout
57722025-11-13T08:02:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
57732025-11-13T08:02:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
57742025-11-13T08:02:32.648ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57752025-11-13T08:02:32.649ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
57762025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 8 reason = Timeout
57772025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57782025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
57792025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
57802025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 8 reason = Timeout
57812025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
57822025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 15 reason = Timeout
57832025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
57842025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 8 reason = Timeout
57852025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
57862025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 15 reason = Timeout
57872025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
57882025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 8 reason = Timeout
57892025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
57902025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 8 reason = Timeout
57912025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
57922025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
57932025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
57942025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
57952025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
57962025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
57972025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
57982025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
57992025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
58002025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
58012025-11-13T08:02:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6161, tv_nsec: 2090966 } })
58022025-11-13T08:02:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
58032025-11-13T08:02:32.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58042025-11-13T08:02:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
58052025-11-13T08:02:32.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58062025-11-13T08:02:32.651ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58072025-11-13T08:02:32.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58082025-11-13T08:02:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
58092025-11-13T08:02:32.653ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
58102025-11-13T08:02:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
58112025-11-13T08:02:32.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
58122025-11-13T08:02:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
58132025-11-13T08:02:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
58142025-11-13T08:02:32.655ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58152025-11-13T08:02:32.655ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58162025-11-13T08:02:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58172025-11-13T08:02:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
58182025-11-13T08:02:32.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
58192025-11-13T08:02:32.657ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
58202025-11-13T08:02:32.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
58212025-11-13T08:02:32.658ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58222025-11-13T08:02:32.660ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 18.172827ms iteration = 8 status = 0
58232025-11-13T08:02:32.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
58242025-11-13T08:02:32.661ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
58252025-11-13T08:02:32.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
58262025-11-13T08:02:32.664ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6161, tv_nsec: 16587800 } })
58272025-11-13T08:02:32.664ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
58282025-11-13T08:02:32.665ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58292025-11-13T08:02:32.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
58302025-11-13T08:02:32.666ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
58312025-11-13T08:02:32.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
58322025-11-13T08:02:32.670ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
58332025-11-13T08:02:32.670ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 27.136403ms iteration = 8 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
58342025-11-13T08:02:32.675ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58352025-11-13T08:02:32.679ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
58362025-11-13T08:02:32.680ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
58372025-11-13T08:02:32.680ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 31.125772ms iteration = 15 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
58382025-11-13T08:02:32.694ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
58392025-11-13T08:02:32.694ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 75.419823ms iteration = 22 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
58402025-11-13T08:02:32.719ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58412025-11-13T08:02:32.722ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
58422025-11-13T08:02:32.722ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
58432025-11-13T08:02:32.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
58442025-11-13T08:02:32.725ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
58452025-11-13T08:02:32.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
58462025-11-13T08:02:32.728ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58472025-11-13T08:02:32.729ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 79.978703ms iteration = 8 status = {"errors":[],"finish_invoked_ok":[]}
58482025-11-13T08:02:32.729ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
58492025-11-13T08:02:32.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
58502025-11-13T08:02:32.730ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
58512025-11-13T08:02:32.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
58522025-11-13T08:02:32.731ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58532025-11-13T08:02:32.731ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 88.572931ms iteration = 22 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
58542025-11-13T08:02:32.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
58552025-11-13T08:02:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
58562025-11-13T08:02:32.732ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
58572025-11-13T08:02:32.732ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 89.835206ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
58582025-11-13T08:02:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
58592025-11-13T08:02:32.734ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
58602025-11-13T08:02:32.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
58612025-11-13T08:02:32.736ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
58622025-11-13T08:02:32.736ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
58632025-11-13T08:02:32.736ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
58642025-11-13T08:02:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
58652025-11-13T08:02:32.737ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 88.230205ms iteration = 8 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
58662025-11-13T08:02:32.737ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
58672025-11-13T08:02:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
58682025-11-13T08:02:32.740ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58692025-11-13T08:02:32.743ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 94.875185ms iteration = 8 status = {"errors":[],"garbage_collect_requested":[]}
58702025-11-13T08:02:32.743ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
58712025-11-13T08:02:32.743ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
58722025-11-13T08:02:32.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
58732025-11-13T08:02:32.746ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 97.537548ms iteration = 8 status = {"errors":[],"requests_created_ok":[]}
58742025-11-13T08:02:32.748ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 99.687409ms iteration = 15 status = "NoSitrep"
58752025-11-13T08:02:32.754ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
58762025-11-13T08:02:32.754ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
58772025-11-13T08:02:32.754ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7846ac77-0b4e-47cc-94ea-430328387c5d", "content-length": "26", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
58782025-11-13T08:02:32.754ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
58792025-11-13T08:02:32.760ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
58802025-11-13T08:02:32.777ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
58812025-11-13T08:02:32.779ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 172.650504ms iteration = 8 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
58822025-11-13T08:02:32.805ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 156.450365ms iteration = 8 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
58832025-11-13T08:02:32.806ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
58842025-11-13T08:02:32.806ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 187.601038ms iteration = 12 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
58852025-11-13T08:02:32.808ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
58862025-11-13T08:02:32.809ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
58872025-11-13T08:02:32.809ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 166.073015ms iteration = 8 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
58882025-11-13T08:02:32.809ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
58892025-11-13T08:02:32.809ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fc71e02-3390-4ac2-91c7-8ea338bf354d", "content-length": "78", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
58902025-11-13T08:02:32.810ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
58912025-11-13T08:02:32.810ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
58922025-11-13T08:02:32.810ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
58932025-11-13T08:02:32.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
58942025-11-13T08:02:32.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
58952025-11-13T08:02:32.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
58962025-11-13T08:02:32.812ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
58972025-11-13T08:02:32.812ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
58982025-11-13T08:02:32.812ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
58992025-11-13T08:02:32.816ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
59002025-11-13T08:02:32.816ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
59012025-11-13T08:02:32.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
59022025-11-13T08:02:32.816ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
59032025-11-13T08:02:32.816ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
59042025-11-13T08:02:32.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
59052025-11-13T08:02:32.817ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
59062025-11-13T08:02:32.817ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
59072025-11-13T08:02:32.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
59082025-11-13T08:02:32.819ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
59092025-11-13T08:02:32.819ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
59102025-11-13T08:02:32.819ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c45c69c-3cae-4299-af34-b2c98128e372", "content-length": "81", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
59112025-11-13T08:02:32.866ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
59122025-11-13T08:02:32.866ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
59132025-11-13T08:02:32.867ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "127f10a0-a5de-4dcc-88f6-345ce232e344", "content-length": "81", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
59142025-11-13T08:02:32.871ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59152025-11-13T08:02:32.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
59162025-11-13T08:02:32.874ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 266.865967ms iteration = 8 status = {}
59172025-11-13T08:02:32.877ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
59182025-11-13T08:02:32.877ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
59192025-11-13T08:02:32.877ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4abdc7b2-a997-4c49-82ff-c96d3e4a4e43", "content-length": "372", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
59202025-11-13T08:02:32.877ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59212025-11-13T08:02:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59222025-11-13T08:02:32.879ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
59232025-11-13T08:02:32.879ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
59242025-11-13T08:02:32.879ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "c96c9788-c275-40d7-bd70-1b6513ca1614", "content-length": "81", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
59252025-11-13T08:02:32.881ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 232.882368ms iteration = 8 status = {"disabled":false,"errors":[],"sps":[]}
59262025-11-13T08:02:32.882ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
59272025-11-13T08:02:32.882ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
59282025-11-13T08:02:32.883ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "91162278-96ce-495c-9f26-f92c2595c01c", "content-length": "26", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
59292025-11-13T08:02:32.883ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
59302025-11-13T08:02:32.884ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
59312025-11-13T08:02:32.893ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 274.630369ms iteration = 14 status = {"physical_disks_added":0}
59322025-11-13T08:02:32.916ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6161, tv_nsec: 268620415 } })
59332025-11-13T08:02:32.926ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59342025-11-13T08:02:32.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
59352025-11-13T08:02:32.928ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59362025-11-13T08:02:32.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
59372025-11-13T08:02:32.931ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59382025-11-13T08:02:32.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
59392025-11-13T08:02:32.932ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59402025-11-13T08:02:32.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
59412025-11-13T08:02:32.935ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
59422025-11-13T08:02:32.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
59432025-11-13T08:02:32.936ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
59442025-11-13T08:02:32.936ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
59452025-11-13T08:02:32.937ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fb241a88-198c-46c1-9c46-5db2ec6bd067", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
59462025-11-13T08:02:32.937ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59472025-11-13T08:02:32.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59482025-11-13T08:02:32.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59492025-11-13T08:02:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59502025-11-13T08:02:32.942ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
59512025-11-13T08:02:32.942ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
59522025-11-13T08:02:32.945ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
59532025-11-13T08:02:32.945ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
59542025-11-13T08:02:32.945ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "17a01d00-e85f-47c1-ae81-bfb44e1cb950", "date": "Thu, 13 Nov 2025 08:02:32 GMT"} })
59552025-11-13T08:02:32.947ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 304.363783ms iteration = 12 status = {}
59562025-11-13T08:02:32.978ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59572025-11-13T08:02:32.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59582025-11-13T08:02:32.981ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
59592025-11-13T08:02:32.981ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59602025-11-13T08:02:32.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59612025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
59622025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
59632025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
59642025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
59652025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59662025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59672025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59682025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
59692025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
59702025-11-13T08:02:32.997ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
59712025-11-13T08:02:32.998ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
59722025-11-13T08:02:32.998ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59732025-11-13T08:02:32.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59742025-11-13T08:02:33.001ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
59752025-11-13T08:02:33.001ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
59762025-11-13T08:02:33.040ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
59772025-11-13T08:02:33.041ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "ced6a87f-7f23-4528-8b82-153c105388f7", "content-length": "691", "date": "Thu, 13 Nov 2025 08:02:33 GMT"} })
59782025-11-13T08:02:33.041ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
59792025-11-13T08:02:33.041ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
59802025-11-13T08:02:33.041ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
59812025-11-13T08:02:33.042ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 399.79425ms iteration = 8 status = {}
59822025-11-13T08:02:33.268ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "36258a64-6feb-4a28-8f97-e890ccd24ac0", "date": "Thu, 13 Nov 2025 08:02:33 GMT"} })
59832025-11-13T08:02:33.268ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
59842025-11-13T08:02:33.268ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
59852025-11-13T08:02:33.268ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "87142311-263b-411d-bd79-b9972f521549", "content-length": "682", "date": "Thu, 13 Nov 2025 08:02:33 GMT"} })
59862025-11-13T08:02:33.268ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59872025-11-13T08:02:33.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59882025-11-13T08:02:33.272ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59892025-11-13T08:02:33.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59902025-11-13T08:02:33.275ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59912025-11-13T08:02:33.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59922025-11-13T08:02:33.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
59932025-11-13T08:02:33.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
59942025-11-13T08:02:33.283ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 641.136404ms iteration = 8 status = {}
59952025-11-13T08:02:33.420ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 1
59962025-11-13T08:02:33.420ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
59972025-11-13T08:02:33.451ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
59982025-11-13T08:02:33.451ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:35761
59992025-11-13T08:02:33.452ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 2bf40ed8-bad4-4855-a31b-e2dd289f3d73 uri = /metrics/producers version = 1.0.0
60002025-11-13T08:02:33.452ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 2bf40ed8-bad4-4855-a31b-e2dd289f3d73 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
60012025-11-13T08:02:33.452ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 2bf40ed8-bad4-4855-a31b-e2dd289f3d73 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
60022025-11-13T08:02:33.502ZINFOnexus: assigned collector to new producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d file = nexus/src/app/oximeter.rs:76 producer_id = bcb58129-79cc-406c-aa16-455a5cd9f031
60032025-11-13T08:02:33.502ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 50306 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:35761 req_id = 2bf40ed8-bad4-4855-a31b-e2dd289f3d73 response_code = 201 uri = /metrics/producers
60042025-11-13T08:02:33.502ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "2bf40ed8-bad4-4855-a31b-e2dd289f3d73", "content-length": "41", "date": "Thu, 13 Nov 2025 08:02:33 GMT"} })
60052025-11-13T08:02:33.504ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
60062025-11-13T08:02:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 317 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 59abec54-1a59-4125-aef9-d9c694852c54 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
60072025-11-13T08:02:37.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 44 reason = Timeout
60082025-11-13T08:02:37.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60092025-11-13T08:02:37.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60102025-11-13T08:02:37.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60112025-11-13T08:02:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
60122025-11-13T08:02:37.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60132025-11-13T08:02:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60142025-11-13T08:02:37.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
60152025-11-13T08:02:37.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.402425ms iteration = 44 status = {"config_updated":false}
60162025-11-13T08:02:42.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 23 reason = Timeout
60172025-11-13T08:02:42.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 45 reason = Timeout
60182025-11-13T08:02:42.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60192025-11-13T08:02:42.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60202025-11-13T08:02:42.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60212025-11-13T08:02:42.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
60222025-11-13T08:02:42.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60232025-11-13T08:02:42.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60242025-11-13T08:02:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60252025-11-13T08:02:42.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60262025-11-13T08:02:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
60272025-11-13T08:02:42.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60282025-11-13T08:02:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
60292025-11-13T08:02:42.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60302025-11-13T08:02:42.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
60312025-11-13T08:02:42.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.792604ms iteration = 45 status = {"config_updated":false}
60322025-11-13T08:02:42.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60332025-11-13T08:02:42.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
60342025-11-13T08:02:42.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60352025-11-13T08:02:42.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
60362025-11-13T08:02:42.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60372025-11-13T08:02:42.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
60382025-11-13T08:02:42.641ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
60392025-11-13T08:02:42.641ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.142665ms iteration = 23 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
60402025-11-13T08:02:42.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 23 reason = Timeout
60412025-11-13T08:02:42.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60422025-11-13T08:02:42.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60432025-11-13T08:02:42.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
60442025-11-13T08:02:42.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
60452025-11-13T08:02:42.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.604791ms iteration = 23 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
60462025-11-13T08:02:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 322 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 505b5196-a1ab-4886-889e-0986b45f98b9 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
60472025-11-13T08:02:47.617ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 21 reason = Timeout
60482025-11-13T08:02:47.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 46 reason = Timeout
60492025-11-13T08:02:47.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
60502025-11-13T08:02:47.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60512025-11-13T08:02:47.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
60522025-11-13T08:02:47.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60532025-11-13T08:02:47.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60542025-11-13T08:02:47.619ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60552025-11-13T08:02:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
60562025-11-13T08:02:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
60572025-11-13T08:02:47.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60582025-11-13T08:02:47.623ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
60592025-11-13T08:02:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
60602025-11-13T08:02:47.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60612025-11-13T08:02:47.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
60622025-11-13T08:02:47.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.918046ms iteration = 46 status = {"config_updated":false}
60632025-11-13T08:02:47.624ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 036cf7b1-53e4-423a-bec9-1e7a1c588e6a original_time_started = 2025-11-13 08:02:03.980290 UTC
60642025-11-13T08:02:47.625ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.201591ms iteration = 21 status = {"Loaded":{"collection_id":"036cf7b1-53e4-423a-bec9-1e7a1c588e6a","time_loaded":"2025-11-13T08:02:47.617810733Z","time_started":"2025-11-13T08:02:03.980290Z"}}
60652025-11-13T08:02:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 16 reason = Timeout
60662025-11-13T08:02:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 16 reason = Timeout
60672025-11-13T08:02:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
60682025-11-13T08:02:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
60692025-11-13T08:02:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
60702025-11-13T08:02:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
60712025-11-13T08:02:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
60722025-11-13T08:02:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
60732025-11-13T08:02:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60742025-11-13T08:02:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60752025-11-13T08:02:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
60762025-11-13T08:02:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
60772025-11-13T08:02:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
60782025-11-13T08:02:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
60792025-11-13T08:02:47.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
60802025-11-13T08:02:47.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
60812025-11-13T08:02:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.248783ms iteration = 16 status = "NoSitrep"
60822025-11-13T08:02:47.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
60832025-11-13T08:02:47.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
60842025-11-13T08:02:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60852025-11-13T08:02:47.656ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
60862025-11-13T08:02:47.657ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
60872025-11-13T08:02:47.657ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.583488ms iteration = 16 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
60882025-11-13T08:02:52.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 24 reason = Timeout
60892025-11-13T08:02:52.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 47 reason = Timeout
60902025-11-13T08:02:52.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
60912025-11-13T08:02:52.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60922025-11-13T08:02:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
60932025-11-13T08:02:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
60942025-11-13T08:02:52.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60952025-11-13T08:02:52.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60962025-11-13T08:02:52.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
60972025-11-13T08:02:52.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
60982025-11-13T08:02:52.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
60992025-11-13T08:02:52.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61002025-11-13T08:02:52.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61012025-11-13T08:02:52.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61022025-11-13T08:02:52.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61032025-11-13T08:02:52.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
61042025-11-13T08:02:52.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.358914ms iteration = 47 status = {"config_updated":false}
61052025-11-13T08:02:52.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61062025-11-13T08:02:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61072025-11-13T08:02:52.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
61082025-11-13T08:02:52.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
61092025-11-13T08:02:52.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
61102025-11-13T08:02:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 24 reason = Timeout
61112025-11-13T08:02:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61122025-11-13T08:02:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61132025-11-13T08:02:52.645ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
61142025-11-13T08:02:52.645ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.592083ms iteration = 24 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
61152025-11-13T08:02:52.672ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
61162025-11-13T08:02:52.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
61172025-11-13T08:02:52.674ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 31.982579ms iteration = 24 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
61182025-11-13T08:02:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 317 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 642d3104-3698-45d5-a585-d927cf2c7938 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
61192025-11-13T08:02:57.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 48 reason = Timeout
61202025-11-13T08:02:57.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61212025-11-13T08:02:57.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61222025-11-13T08:02:57.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61232025-11-13T08:02:57.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
61242025-11-13T08:02:57.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
61252025-11-13T08:02:57.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
61262025-11-13T08:02:57.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
61272025-11-13T08:02:57.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.746822ms iteration = 48 status = {"config_updated":false}
61282025-11-13T08:03:00.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 98634d3c-eb16-4b0e-a4ce-dde7ab31abb5 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
61292025-11-13T08:03:00.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 98634d3c-eb16-4b0e-a4ce-dde7ab31abb5 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
61302025-11-13T08:03:00.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 98634d3c-eb16-4b0e-a4ce-dde7ab31abb5 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
61312025-11-13T08:03:00.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3046 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = 98634d3c-eb16-4b0e-a4ce-dde7ab31abb5 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
61322025-11-13T08:03:00.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = f36aac22-308f-4f40-82e4-3493712abc4b uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
61332025-11-13T08:03:00.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = f36aac22-308f-4f40-82e4-3493712abc4b roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
61342025-11-13T08:03:00.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = f36aac22-308f-4f40-82e4-3493712abc4b resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
61352025-11-13T08:03:00.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2934 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:53638 req_id = f36aac22-308f-4f40-82e4-3493712abc4b response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
61362025-11-13T08:03:02.606ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 5 reason = Timeout
61372025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 5 reason = Timeout
61382025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 5 reason = Timeout
61392025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
61402025-11-13T08:03:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:53:02.607007497 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
61412025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61422025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
61432025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 5 reason = Timeout
61442025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
61452025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61462025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 9 reason = Timeout
61472025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 9 reason = Timeout
61482025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
61492025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 5 reason = Timeout
61502025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 9 reason = Timeout
61512025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61522025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
61532025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 7 reason = Timeout
61542025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 5 reason = Timeout
61552025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 7 reason = Timeout
61562025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
61572025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
61582025-11-13T08:03:02.607ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6190, tv_nsec: 960016945 } })
61592025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61602025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61612025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
61622025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 6 reason = Timeout
61632025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
61642025-11-13T08:03:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
61652025-11-13T08:03:02.607ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = ExternalNetworking step id = Ensure
61662025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61672025-11-13T08:03:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
61682025-11-13T08:03:02.607ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6190, tv_nsec: 960246130 } })
61692025-11-13T08:03:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
61702025-11-13T08:03:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.244914ms iteration = 5 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
61712025-11-13T08:03:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61722025-11-13T08:03:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
61732025-11-13T08:03:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.469219ms iteration = 5 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
61742025-11-13T08:03:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
61752025-11-13T08:03:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
61762025-11-13T08:03:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
61772025-11-13T08:03:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
61782025-11-13T08:03:02.610ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61792025-11-13T08:03:02.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6190, tv_nsec: 963355541 } })
61802025-11-13T08:03:02.611ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6190, tv_nsec: 964092065 } })
61812025-11-13T08:03:02.611ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:53:02.607007497 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
61822025-11-13T08:03:02.611ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.773493ms iteration = 5 status = {"expiration":"2025-11-13T07:53:02.607007497Z","pruned":[]}
61832025-11-13T08:03:02.612ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
61842025-11-13T08:03:02.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61852025-11-13T08:03:02.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61862025-11-13T08:03:02.613ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61872025-11-13T08:03:02.615ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
61882025-11-13T08:03:02.615ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
61892025-11-13T08:03:02.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
61902025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
61912025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
61922025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.69228ms iteration = 9 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
61932025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 15 reason = Timeout
61942025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 5 reason = Timeout
61952025-11-13T08:03:02.617ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
61962025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 13 reason = Timeout
61972025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
61982025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
61992025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62002025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 25 reason = Timeout
62012025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 49 reason = Timeout
62022025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62032025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 22 reason = Timeout
62042025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
62052025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 13 reason = Timeout
62062025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
62072025-11-13T08:03:02.617ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
62082025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62092025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62102025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62112025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
62122025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
62132025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
62142025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
62152025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
62162025-11-13T08:03:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
62172025-11-13T08:03:02.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62182025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62192025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62202025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62212025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
62222025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
62232025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
62242025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62252025-11-13T08:03:02.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62262025-11-13T08:03:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62272025-11-13T08:03:02.622ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62282025-11-13T08:03:02.623ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62292025-11-13T08:03:02.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
62302025-11-13T08:03:02.626ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
62312025-11-13T08:03:02.626ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
62322025-11-13T08:03:02.626ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
62332025-11-13T08:03:02.626ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62342025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62352025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62362025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62372025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62382025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
62392025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
62402025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
62412025-11-13T08:03:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62422025-11-13T08:03:02.628ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62432025-11-13T08:03:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
62442025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
62452025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
62462025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62472025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
62482025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
62492025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62502025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
62512025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
62522025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62532025-11-13T08:03:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
62542025-11-13T08:03:02.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
62552025-11-13T08:03:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
62562025-11-13T08:03:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
62572025-11-13T08:03:02.631ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62582025-11-13T08:03:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
62592025-11-13T08:03:02.631ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
62602025-11-13T08:03:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
62612025-11-13T08:03:02.632ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
62622025-11-13T08:03:02.633ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 036cf7b1-53e4-423a-bec9-1e7a1c588e6a original_time_started = 2025-11-13 08:02:03.980290 UTC
62632025-11-13T08:03:02.633ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 15.742309ms iteration = 22 status = {"Loaded":{"collection_id":"036cf7b1-53e4-423a-bec9-1e7a1c588e6a","time_loaded":"2025-11-13T08:03:02.617890201Z","time_started":"2025-11-13T08:02:03.980290Z"}}
62642025-11-13T08:03:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
62652025-11-13T08:03:02.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
62662025-11-13T08:03:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
62672025-11-13T08:03:02.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
62682025-11-13T08:03:02.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
62692025-11-13T08:03:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
62702025-11-13T08:03:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
62712025-11-13T08:03:02.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
62722025-11-13T08:03:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
62732025-11-13T08:03:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
62742025-11-13T08:03:02.637ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
62752025-11-13T08:03:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
62762025-11-13T08:03:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
62772025-11-13T08:03:02.638ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
62782025-11-13T08:03:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
62792025-11-13T08:03:02.641ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
62802025-11-13T08:03:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 5 reason = Timeout
62812025-11-13T08:03:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
62822025-11-13T08:03:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 9 reason = Timeout
62832025-11-13T08:03:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
62842025-11-13T08:03:02.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 9 reason = Timeout
62852025-11-13T08:03:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
62862025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
62872025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
62882025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 13 reason = Timeout
62892025-11-13T08:03:02.643ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
62902025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
62912025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 9 reason = Timeout
62922025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
62932025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 25 reason = Timeout
62942025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
62952025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 9 reason = Timeout
62962025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 9 reason = Timeout
62972025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
62982025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
62992025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 9 reason = Timeout
63002025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
63012025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 5 reason = Timeout
63022025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
63032025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
63042025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63052025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
63062025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63072025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
63082025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
63092025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
63102025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63112025-11-13T08:03:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
63122025-11-13T08:03:02.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63132025-11-13T08:03:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63142025-11-13T08:03:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 5 reason = Timeout
63152025-11-13T08:03:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
63162025-11-13T08:03:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
63172025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 5 reason = Timeout
63182025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
63192025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
63202025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 9 reason = Timeout
63212025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
63222025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 5 reason = Timeout
63232025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
63242025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 9 reason = Timeout
63252025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 9 reason = Timeout
63262025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 17 reason = Timeout
63272025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 9 reason = Timeout
63282025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
63292025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
63302025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
63312025-11-13T08:03:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 9 reason = Timeout
63322025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
63332025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 9 reason = Timeout
63342025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63352025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 17 reason = Timeout
63362025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
63372025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 5 reason = Timeout
63382025-11-13T08:03:02.649ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
63392025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
63402025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
63412025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 31.300246ms iteration = 49 status = {"config_updated":false}
63422025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
63432025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
63442025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
63452025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
63462025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
63472025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63482025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
63492025-11-13T08:03:02.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6191, tv_nsec: 1919089 } })
63502025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
63512025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
63522025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
63532025-11-13T08:03:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
63542025-11-13T08:03:02.652ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
63552025-11-13T08:03:02.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
63562025-11-13T08:03:02.654ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63572025-11-13T08:03:02.658ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
63582025-11-13T08:03:02.658ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 10.364084ms iteration = 5 status = {"by_rx":{},"error":null}
63592025-11-13T08:03:02.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
63602025-11-13T08:03:02.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
63612025-11-13T08:03:02.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
63622025-11-13T08:03:02.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63632025-11-13T08:03:02.661ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
63642025-11-13T08:03:02.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
63652025-11-13T08:03:02.664ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
63662025-11-13T08:03:02.664ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
63672025-11-13T08:03:02.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
63682025-11-13T08:03:02.664ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 19.451312ms iteration = 5 status = {"errors":[],"found_port_ok":[]}
63692025-11-13T08:03:02.668ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
63702025-11-13T08:03:02.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
63712025-11-13T08:03:02.677ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
63722025-11-13T08:03:02.685ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63732025-11-13T08:03:02.688ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
63742025-11-13T08:03:02.690ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
63752025-11-13T08:03:02.690ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
63762025-11-13T08:03:02.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
63772025-11-13T08:03:02.697ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
63782025-11-13T08:03:02.699ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
63792025-11-13T08:03:02.706ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
63802025-11-13T08:03:02.706ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 57.325667ms iteration = 9 status = {"errors":[],"requests_created_ok":[]}
63812025-11-13T08:03:02.715ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
63822025-11-13T08:03:02.716ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
63832025-11-13T08:03:02.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
63842025-11-13T08:03:02.723ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63852025-11-13T08:03:02.724ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
63862025-11-13T08:03:02.727ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
63872025-11-13T08:03:02.727ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
63882025-11-13T08:03:02.727ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
63892025-11-13T08:03:02.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
63902025-11-13T08:03:02.728ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
63912025-11-13T08:03:02.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
63922025-11-13T08:03:02.737ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
63932025-11-13T08:03:02.740ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
63942025-11-13T08:03:02.741ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = SledList step id = Fetch
63952025-11-13T08:03:02.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
63962025-11-13T08:03:02.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
63972025-11-13T08:03:02.745ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63982025-11-13T08:03:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
63992025-11-13T08:03:02.750ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
64002025-11-13T08:03:02.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
64012025-11-13T08:03:02.752ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
64022025-11-13T08:03:02.752ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
64032025-11-13T08:03:02.752ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 134.584986ms iteration = 5 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
64042025-11-13T08:03:02.755ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64052025-11-13T08:03:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
64062025-11-13T08:03:02.760ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
64072025-11-13T08:03:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
64082025-11-13T08:03:02.761ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64092025-11-13T08:03:02.764ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
64102025-11-13T08:03:02.766ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64112025-11-13T08:03:02.767ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
64122025-11-13T08:03:02.767ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
64132025-11-13T08:03:02.767ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
64142025-11-13T08:03:02.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
64152025-11-13T08:03:02.769ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 120.368227ms iteration = 17 status = "NoSitrep"
64162025-11-13T08:03:02.770ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
64172025-11-13T08:03:02.770ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
64182025-11-13T08:03:02.770ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64192025-11-13T08:03:02.770ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
64202025-11-13T08:03:02.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
64212025-11-13T08:03:02.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
64222025-11-13T08:03:02.771ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
64232025-11-13T08:03:02.773ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64242025-11-13T08:03:02.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64252025-11-13T08:03:02.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
64262025-11-13T08:03:02.775ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
64272025-11-13T08:03:02.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
64282025-11-13T08:03:02.777ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
64292025-11-13T08:03:02.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64302025-11-13T08:03:02.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64312025-11-13T08:03:02.778ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64322025-11-13T08:03:02.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64332025-11-13T08:03:02.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
64342025-11-13T08:03:02.778ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
64352025-11-13T08:03:02.779ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
64362025-11-13T08:03:02.779ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
64372025-11-13T08:03:02.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
64382025-11-13T08:03:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
64392025-11-13T08:03:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
64402025-11-13T08:03:02.779ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64412025-11-13T08:03:02.779ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6191, tv_nsec: 132250462 } })
64422025-11-13T08:03:02.780ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
64432025-11-13T08:03:02.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
64442025-11-13T08:03:02.783ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
64452025-11-13T08:03:02.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
64462025-11-13T08:03:02.786ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 137.85686ms iteration = 5 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
64472025-11-13T08:03:02.791ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
64482025-11-13T08:03:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
64492025-11-13T08:03:02.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64502025-11-13T08:03:02.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64512025-11-13T08:03:02.794ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64522025-11-13T08:03:02.794ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64532025-11-13T08:03:02.794ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 146.046571ms iteration = 9 status = {"errors":[],"garbage_collect_requested":[]}
64542025-11-13T08:03:02.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
64552025-11-13T08:03:02.797ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64562025-11-13T08:03:02.797ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64572025-11-13T08:03:02.797ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64582025-11-13T08:03:02.799ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
64592025-11-13T08:03:02.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
64602025-11-13T08:03:02.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
64612025-11-13T08:03:02.801ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
64622025-11-13T08:03:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
64632025-11-13T08:03:02.802ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
64642025-11-13T08:03:02.802ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
64652025-11-13T08:03:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
64662025-11-13T08:03:02.802ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
64672025-11-13T08:03:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
64682025-11-13T08:03:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
64692025-11-13T08:03:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
64702025-11-13T08:03:02.803ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
64712025-11-13T08:03:02.803ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
64722025-11-13T08:03:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
64732025-11-13T08:03:02.804ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
64742025-11-13T08:03:02.804ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 155.183411ms iteration = 17 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
64752025-11-13T08:03:02.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
64762025-11-13T08:03:02.805ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
64772025-11-13T08:03:02.805ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64782025-11-13T08:03:02.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
64792025-11-13T08:03:02.805ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 163.033966ms iteration = 9 status = 0
64802025-11-13T08:03:02.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
64812025-11-13T08:03:02.810ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
64822025-11-13T08:03:02.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
64832025-11-13T08:03:02.810ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
64842025-11-13T08:03:02.810ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
64852025-11-13T08:03:02.810ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
64862025-11-13T08:03:02.810ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 202.882129ms iteration = 5 status = {"generation":2}
64872025-11-13T08:03:02.812ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64882025-11-13T08:03:02.812ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
64892025-11-13T08:03:02.812ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64902025-11-13T08:03:02.813ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
64912025-11-13T08:03:02.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
64922025-11-13T08:03:02.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
64932025-11-13T08:03:02.816ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
64942025-11-13T08:03:02.816ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
64952025-11-13T08:03:02.816ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
64962025-11-13T08:03:02.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
64972025-11-13T08:03:02.830ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
64982025-11-13T08:03:02.830ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
64992025-11-13T08:03:02.830ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
65002025-11-13T08:03:02.830ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 223.834141ms iteration = 5 status = {"generation":1}
65012025-11-13T08:03:02.844ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
65022025-11-13T08:03:02.844ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 226.202057ms iteration = 25 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
65032025-11-13T08:03:02.846ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 197.550364ms iteration = 9 status = {"errors":[],"finish_invoked_ok":[]}
65042025-11-13T08:03:02.848ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65052025-11-13T08:03:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65062025-11-13T08:03:02.855ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
65072025-11-13T08:03:02.855ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
65082025-11-13T08:03:02.855ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 212.675461ms iteration = 5 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
65092025-11-13T08:03:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 217.411374ms iteration = 25 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
65102025-11-13T08:03:02.869ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 220.455514ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65112025-11-13T08:03:02.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65122025-11-13T08:03:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65132025-11-13T08:03:02.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65142025-11-13T08:03:02.879ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
65152025-11-13T08:03:02.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65162025-11-13T08:03:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65172025-11-13T08:03:02.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65182025-11-13T08:03:02.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
65192025-11-13T08:03:02.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65202025-11-13T08:03:02.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65212025-11-13T08:03:02.896ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
65222025-11-13T08:03:02.896ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 253.446062ms iteration = 9 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
65232025-11-13T08:03:02.908ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
65242025-11-13T08:03:02.908ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
65252025-11-13T08:03:02.908ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 265.793415ms iteration = 9 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
65262025-11-13T08:03:02.918ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
65272025-11-13T08:03:02.918ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 300.671301ms iteration = 13 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
65282025-11-13T08:03:02.923ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
65292025-11-13T08:03:02.923ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 316.534533ms iteration = 5 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
65302025-11-13T08:03:02.927ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 284.165137ms iteration = 9 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
65312025-11-13T08:03:02.929ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
65322025-11-13T08:03:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
65332025-11-13T08:03:02.948ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
65342025-11-13T08:03:02.949ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad36ab2f-2322-42db-a221-b165e813f13b", "content-length": "78", "date": "Thu, 13 Nov 2025 08:03:02 GMT"} })
65352025-11-13T08:03:02.949ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
65362025-11-13T08:03:02.949ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
65372025-11-13T08:03:02.949ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
65382025-11-13T08:03:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
65392025-11-13T08:03:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
65402025-11-13T08:03:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
65412025-11-13T08:03:02.951ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
65422025-11-13T08:03:02.952ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
65432025-11-13T08:03:02.952ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
65442025-11-13T08:03:02.956ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
65452025-11-13T08:03:02.956ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
65462025-11-13T08:03:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
65472025-11-13T08:03:02.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
65482025-11-13T08:03:02.957ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
65492025-11-13T08:03:02.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
65502025-11-13T08:03:02.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
65512025-11-13T08:03:02.957ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
65522025-11-13T08:03:02.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
65532025-11-13T08:03:02.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65542025-11-13T08:03:02.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65552025-11-13T08:03:02.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65562025-11-13T08:03:02.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65572025-11-13T08:03:02.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65582025-11-13T08:03:02.994ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = DeployNexusRecords step id = Ensure
65592025-11-13T08:03:02.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65602025-11-13T08:03:02.994ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
65612025-11-13T08:03:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
65622025-11-13T08:03:02.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65632025-11-13T08:03:02.995ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
65642025-11-13T08:03:02.995ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65652025-11-13T08:03:02.996ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc699ccb-9ca5-4ba5-82d6-79f109421af3", "content-length": "26", "date": "Thu, 13 Nov 2025 08:03:02 GMT"} })
65662025-11-13T08:03:02.996ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65672025-11-13T08:03:02.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65682025-11-13T08:03:02.998ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
65692025-11-13T08:03:02.999ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
65702025-11-13T08:03:02.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65712025-11-13T08:03:02.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65722025-11-13T08:03:03.000ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
65732025-11-13T08:03:03.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65742025-11-13T08:03:03.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65752025-11-13T08:03:03.005ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
65762025-11-13T08:03:03.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
65772025-11-13T08:03:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
65782025-11-13T08:03:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
65792025-11-13T08:03:03.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
65802025-11-13T08:03:03.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
65812025-11-13T08:03:03.013ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 364.65662ms iteration = 9 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
65822025-11-13T08:03:03.026ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
65832025-11-13T08:03:03.026ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
65842025-11-13T08:03:03.027ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "b167b63b-d86c-44fc-a031-7ee33701d3e4", "content-length": "81", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
65852025-11-13T08:03:03.034ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
65862025-11-13T08:03:03.034ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
65872025-11-13T08:03:03.034ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "953d50c8-cc8d-4dbb-bb77-e1308c8c4161", "content-length": "81", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
65882025-11-13T08:03:03.036ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
65892025-11-13T08:03:03.036ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
65902025-11-13T08:03:03.037ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c7013b5-6ae6-4178-ad4a-0400b9116e1a", "content-length": "81", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
65912025-11-13T08:03:03.037ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
65922025-11-13T08:03:03.037ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
65932025-11-13T08:03:03.037ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "622a722f-1c31-494a-950b-de2575cb3c67", "content-length": "26", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
65942025-11-13T08:03:03.037ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
65952025-11-13T08:03:03.046ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
65962025-11-13T08:03:03.051ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
65972025-11-13T08:03:03.052ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 403.206987ms iteration = 9 status = {"disabled":false,"errors":[],"sps":[]}
65982025-11-13T08:03:03.052ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 434.38187ms iteration = 15 status = {"physical_disks_added":0}
65992025-11-13T08:03:03.061ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
66002025-11-13T08:03:03.068ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 461.704317ms iteration = 9 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
66012025-11-13T08:03:03.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66022025-11-13T08:03:03.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66032025-11-13T08:03:03.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
66042025-11-13T08:03:03.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66052025-11-13T08:03:03.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66062025-11-13T08:03:03.085ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66072025-11-13T08:03:03.086ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
66082025-11-13T08:03:03.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
66092025-11-13T08:03:03.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66102025-11-13T08:03:03.088ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66112025-11-13T08:03:03.093ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
66122025-11-13T08:03:03.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66132025-11-13T08:03:03.093ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66142025-11-13T08:03:03.093ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66152025-11-13T08:03:03.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66162025-11-13T08:03:03.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
66172025-11-13T08:03:03.096ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66182025-11-13T08:03:03.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66192025-11-13T08:03:03.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66202025-11-13T08:03:03.099ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
66212025-11-13T08:03:03.099ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
66222025-11-13T08:03:03.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
66232025-11-13T08:03:03.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66242025-11-13T08:03:03.104ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
66252025-11-13T08:03:03.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66262025-11-13T08:03:03.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66272025-11-13T08:03:03.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66282025-11-13T08:03:03.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66292025-11-13T08:03:03.109ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
66302025-11-13T08:03:03.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66312025-11-13T08:03:03.115ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
66322025-11-13T08:03:03.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66332025-11-13T08:03:03.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66342025-11-13T08:03:03.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66352025-11-13T08:03:03.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66362025-11-13T08:03:03.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66372025-11-13T08:03:03.129ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
66382025-11-13T08:03:03.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66392025-11-13T08:03:03.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66402025-11-13T08:03:03.141ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
66412025-11-13T08:03:03.141ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66422025-11-13T08:03:03.141ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa35d77e-8206-4f01-af08-dcf64f909934", "content-length": "372", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
66432025-11-13T08:03:03.141ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66442025-11-13T08:03:03.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66452025-11-13T08:03:03.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66462025-11-13T08:03:03.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66472025-11-13T08:03:03.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66482025-11-13T08:03:03.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66492025-11-13T08:03:03.157ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = SledAgent step id = Ensure
66502025-11-13T08:03:03.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66512025-11-13T08:03:03.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
66522025-11-13T08:03:03.167ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66532025-11-13T08:03:03.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66542025-11-13T08:03:03.169ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66552025-11-13T08:03:03.173ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
66562025-11-13T08:03:03.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66572025-11-13T08:03:03.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66582025-11-13T08:03:03.175ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66592025-11-13T08:03:03.176ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66602025-11-13T08:03:03.178ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66612025-11-13T08:03:03.178ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66622025-11-13T08:03:03.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66632025-11-13T08:03:03.186ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
66642025-11-13T08:03:03.186ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66652025-11-13T08:03:03.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66662025-11-13T08:03:03.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66672025-11-13T08:03:03.194ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
66682025-11-13T08:03:03.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66692025-11-13T08:03:03.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66702025-11-13T08:03:03.197ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66712025-11-13T08:03:03.197ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66722025-11-13T08:03:03.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66732025-11-13T08:03:03.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66742025-11-13T08:03:03.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66752025-11-13T08:03:03.206ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
66762025-11-13T08:03:03.206ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 563.565829ms iteration = 5 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
66772025-11-13T08:03:03.208ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
66782025-11-13T08:03:03.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66792025-11-13T08:03:03.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66802025-11-13T08:03:03.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
66812025-11-13T08:03:03.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
66822025-11-13T08:03:03.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66832025-11-13T08:03:03.215ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
66842025-11-13T08:03:03.216ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
66852025-11-13T08:03:03.216ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
66862025-11-13T08:03:03.216ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c622464c-0f40-4852-b094-4fe67ac896e1", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
66872025-11-13T08:03:03.216ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66882025-11-13T08:03:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66892025-11-13T08:03:03.219ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66902025-11-13T08:03:03.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66912025-11-13T08:03:03.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
66922025-11-13T08:03:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
66932025-11-13T08:03:03.226ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
66942025-11-13T08:03:03.226ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
66952025-11-13T08:03:03.227ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "06397db7-0928-49b5-a671-92823f708a50", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
66962025-11-13T08:03:03.228ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 585.93305ms iteration = 13 status = {}
66972025-11-13T08:03:03.236ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
66982025-11-13T08:03:03.237ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "98f3c3fe-d840-4ea6-b6ef-5d9e81536b39", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
66992025-11-13T08:03:03.237ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
67002025-11-13T08:03:03.241ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
67012025-11-13T08:03:03.241ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 592.234733ms iteration = 5 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
67022025-11-13T08:03:03.241ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 14 reason = Dependency
67032025-11-13T08:03:03.241ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
67042025-11-13T08:03:03.241ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67052025-11-13T08:03:03.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67062025-11-13T08:03:03.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67072025-11-13T08:03:03.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67082025-11-13T08:03:03.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67092025-11-13T08:03:03.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67102025-11-13T08:03:03.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67112025-11-13T08:03:03.277ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
67122025-11-13T08:03:03.277ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67132025-11-13T08:03:03.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67142025-11-13T08:03:03.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67152025-11-13T08:03:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67162025-11-13T08:03:03.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67172025-11-13T08:03:03.284ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 676.9972ms iteration = 9 status = {}
67182025-11-13T08:03:03.286ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
67192025-11-13T08:03:03.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67202025-11-13T08:03:03.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67212025-11-13T08:03:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67222025-11-13T08:03:03.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67232025-11-13T08:03:03.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67242025-11-13T08:03:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
67252025-11-13T08:03:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67262025-11-13T08:03:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67272025-11-13T08:03:03.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67282025-11-13T08:03:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67292025-11-13T08:03:03.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67302025-11-13T08:03:03.299ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67312025-11-13T08:03:03.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67322025-11-13T08:03:03.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67332025-11-13T08:03:03.300ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67342025-11-13T08:03:03.301ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67352025-11-13T08:03:03.303ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
67362025-11-13T08:03:03.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67372025-11-13T08:03:03.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67382025-11-13T08:03:03.304ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
67392025-11-13T08:03:03.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67402025-11-13T08:03:03.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67412025-11-13T08:03:03.313ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
67422025-11-13T08:03:03.313ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "59469bae-811c-4822-b483-457b3a472434", "content-length": "691", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
67432025-11-13T08:03:03.313ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
67442025-11-13T08:03:03.313ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
67452025-11-13T08:03:03.313ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
67462025-11-13T08:03:03.316ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 673.632134ms iteration = 9 status = {}
67472025-11-13T08:03:03.351ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6191, tv_nsec: 703958030 } })
67482025-11-13T08:03:03.355ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
67492025-11-13T08:03:03.355ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
67502025-11-13T08:03:03.355ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 8c60f9fa-ffbe-40a4-afcc-0643cea1f6c3 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
67512025-11-13T08:03:03.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67522025-11-13T08:03:03.356ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67532025-11-13T08:03:03.357ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67542025-11-13T08:03:03.358ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
67552025-11-13T08:03:03.358ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "8577715a-4612-4501-9d33-b6b3dbcd8075", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
67562025-11-13T08:03:03.358ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
67572025-11-13T08:03:03.360ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "95680145-bd7c-416d-a4a9-c6004b9d4a0b", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
67582025-11-13T08:03:03.360ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
67592025-11-13T08:03:03.363ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
67602025-11-13T08:03:03.363ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 756.062783ms iteration = 7 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
67612025-11-13T08:03:03.363ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
67622025-11-13T08:03:03.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
67632025-11-13T08:03:03.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
67642025-11-13T08:03:03.367ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
67652025-11-13T08:03:03.367ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
67662025-11-13T08:03:03.367ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 749.598866ms iteration = 13 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
67672025-11-13T08:03:03.371ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
67682025-11-13T08:03:03.371ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "e35e3f23-43ac-46d1-af3c-3595a326ceff", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
67692025-11-13T08:03:03.372ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
67702025-11-13T08:03:03.374ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "60872fde-04b1-4ddd-9319-6f58d364f9f9", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
67712025-11-13T08:03:03.374ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
67722025-11-13T08:03:03.377ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "833b3858-8e82-45ef-acf2-9161a5b39d5d", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
67732025-11-13T08:03:03.377ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
67742025-11-13T08:03:03.380ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
67752025-11-13T08:03:03.380ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 773.021516ms iteration = 7 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
67762025-11-13T08:03:03.397ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
67772025-11-13T08:03:03.400ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "31b4ba50-f2ac-4b12-bfa5-4ac806deab1c", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
67782025-11-13T08:03:03.402ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = FirewallRules step id = Ensure
67792025-11-13T08:03:03.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67802025-11-13T08:03:03.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67812025-11-13T08:03:03.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67822025-11-13T08:03:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67832025-11-13T08:03:03.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67842025-11-13T08:03:03.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67852025-11-13T08:03:03.410ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67862025-11-13T08:03:03.410ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67872025-11-13T08:03:03.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67882025-11-13T08:03:03.411ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
67892025-11-13T08:03:03.411ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
67902025-11-13T08:03:03.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67912025-11-13T08:03:03.412ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f77f6ba-d89d-4624-be30-3cac57cb28ef", "content-length": "372", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
67922025-11-13T08:03:03.412ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
67932025-11-13T08:03:03.412ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
67942025-11-13T08:03:03.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67952025-11-13T08:03:03.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
67962025-11-13T08:03:03.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67972025-11-13T08:03:03.421ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
67982025-11-13T08:03:03.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
67992025-11-13T08:03:03.421ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68002025-11-13T08:03:03.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68012025-11-13T08:03:03.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68022025-11-13T08:03:03.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68032025-11-13T08:03:03.425ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68042025-11-13T08:03:03.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68052025-11-13T08:03:03.428ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
68062025-11-13T08:03:03.428ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
68072025-11-13T08:03:03.429ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "8f7875f5-0bca-4dcb-9af9-03c3dd3e9d31", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
68082025-11-13T08:03:03.429ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68092025-11-13T08:03:03.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68102025-11-13T08:03:03.431ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
68112025-11-13T08:03:03.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68122025-11-13T08:03:03.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68132025-11-13T08:03:03.432ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68142025-11-13T08:03:03.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68152025-11-13T08:03:03.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68162025-11-13T08:03:03.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68172025-11-13T08:03:03.436ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
68182025-11-13T08:03:03.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
68192025-11-13T08:03:03.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68202025-11-13T08:03:03.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68212025-11-13T08:03:03.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68222025-11-13T08:03:03.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68232025-11-13T08:03:03.439ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
68242025-11-13T08:03:03.439ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
68252025-11-13T08:03:03.441ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68262025-11-13T08:03:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68272025-11-13T08:03:03.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68282025-11-13T08:03:03.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68292025-11-13T08:03:03.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68302025-11-13T08:03:03.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68312025-11-13T08:03:03.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68322025-11-13T08:03:03.447ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "36753847-2024-4930-a8bc-cfda09afbab5", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
68332025-11-13T08:03:03.450ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 209.56115ms iteration = 14 status = {}
68342025-11-13T08:03:03.452ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
68352025-11-13T08:03:03.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68362025-11-13T08:03:03.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68372025-11-13T08:03:03.454ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68382025-11-13T08:03:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68392025-11-13T08:03:03.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68402025-11-13T08:03:03.457ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68412025-11-13T08:03:03.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68422025-11-13T08:03:03.460ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68432025-11-13T08:03:03.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68442025-11-13T08:03:03.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68452025-11-13T08:03:03.465ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68462025-11-13T08:03:03.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68472025-11-13T08:03:03.466ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68482025-11-13T08:03:03.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68492025-11-13T08:03:03.467ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68502025-11-13T08:03:03.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68512025-11-13T08:03:03.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68522025-11-13T08:03:03.469ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
68532025-11-13T08:03:03.469ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68542025-11-13T08:03:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68552025-11-13T08:03:03.475ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
68562025-11-13T08:03:03.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68572025-11-13T08:03:03.475ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68582025-11-13T08:03:03.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68592025-11-13T08:03:03.478ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68602025-11-13T08:03:03.479ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68612025-11-13T08:03:03.480ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68622025-11-13T08:03:03.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68632025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
68642025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
68652025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
68662025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
68672025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68682025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68692025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68702025-11-13T08:03:03.484ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68712025-11-13T08:03:03.485ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
68722025-11-13T08:03:03.485ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
68732025-11-13T08:03:03.485ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
68742025-11-13T08:03:03.485ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
68752025-11-13T08:03:03.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
68762025-11-13T08:03:03.487ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
68772025-11-13T08:03:03.487ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
68782025-11-13T08:03:03.487ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
68792025-11-13T08:03:03.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68802025-11-13T08:03:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68812025-11-13T08:03:03.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68822025-11-13T08:03:03.497ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68832025-11-13T08:03:03.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68842025-11-13T08:03:03.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68852025-11-13T08:03:03.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68862025-11-13T08:03:03.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68872025-11-13T08:03:03.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68882025-11-13T08:03:03.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68892025-11-13T08:03:03.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68902025-11-13T08:03:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68912025-11-13T08:03:03.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68922025-11-13T08:03:03.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68932025-11-13T08:03:03.508ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68942025-11-13T08:03:03.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68952025-11-13T08:03:03.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68962025-11-13T08:03:03.515ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
68972025-11-13T08:03:03.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
68982025-11-13T08:03:03.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
68992025-11-13T08:03:03.518ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69002025-11-13T08:03:03.518ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69012025-11-13T08:03:03.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69022025-11-13T08:03:03.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69032025-11-13T08:03:03.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69042025-11-13T08:03:03.526ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
69052025-11-13T08:03:03.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69062025-11-13T08:03:03.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69072025-11-13T08:03:03.532ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
69082025-11-13T08:03:03.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69092025-11-13T08:03:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69102025-11-13T08:03:03.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69112025-11-13T08:03:03.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
69122025-11-13T08:03:03.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69132025-11-13T08:03:03.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69142025-11-13T08:03:03.538ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
69152025-11-13T08:03:03.538ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
69162025-11-13T08:03:03.538ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
69172025-11-13T08:03:03.605ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
69182025-11-13T08:03:03.605ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
69192025-11-13T08:03:03.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69202025-11-13T08:03:03.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69212025-11-13T08:03:03.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69222025-11-13T08:03:03.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69232025-11-13T08:03:03.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69242025-11-13T08:03:03.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
69252025-11-13T08:03:03.643ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
69262025-11-13T08:03:03.648ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "d33eb5d8-3375-4c92-8a44-8c301e36d678", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
69272025-11-13T08:03:03.649ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
69282025-11-13T08:03:03.649ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = Dns step id = Ensure
69292025-11-13T08:03:03.649ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
69302025-11-13T08:03:03.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69312025-11-13T08:03:03.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69322025-11-13T08:03:03.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69332025-11-13T08:03:03.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69342025-11-13T08:03:03.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69352025-11-13T08:03:03.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69362025-11-13T08:03:03.656ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
69372025-11-13T08:03:03.656ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
69382025-11-13T08:03:03.656ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
69392025-11-13T08:03:03.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69402025-11-13T08:03:03.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69412025-11-13T08:03:03.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69422025-11-13T08:03:03.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69432025-11-13T08:03:03.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69442025-11-13T08:03:03.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69452025-11-13T08:03:03.663ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
69462025-11-13T08:03:03.663ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
69472025-11-13T08:03:03.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69482025-11-13T08:03:03.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69492025-11-13T08:03:03.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69502025-11-13T08:03:03.668ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69512025-11-13T08:03:03.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69522025-11-13T08:03:03.668ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69532025-11-13T08:03:03.671ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
69542025-11-13T08:03:03.671ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
69552025-11-13T08:03:03.671ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
69562025-11-13T08:03:03.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69572025-11-13T08:03:03.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69582025-11-13T08:03:03.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69592025-11-13T08:03:03.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69602025-11-13T08:03:03.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69612025-11-13T08:03:03.678ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69622025-11-13T08:03:03.721ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
69632025-11-13T08:03:03.721ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
69642025-11-13T08:03:03.721ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
69652025-11-13T08:03:03.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69662025-11-13T08:03:03.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69672025-11-13T08:03:03.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69682025-11-13T08:03:03.727ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69692025-11-13T08:03:03.727ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69702025-11-13T08:03:03.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69712025-11-13T08:03:03.754ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "bf326a43-7eb0-4da8-92a9-03c5edd61d5f", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
69722025-11-13T08:03:03.754ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
69732025-11-13T08:03:03.754ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
69742025-11-13T08:03:03.755ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f02cae2-cc8e-40aa-bb54-5aa7f43fbd0b", "content-length": "682", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
69752025-11-13T08:03:03.755ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69762025-11-13T08:03:03.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69772025-11-13T08:03:03.758ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69782025-11-13T08:03:03.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69792025-11-13T08:03:03.761ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
69802025-11-13T08:03:03.761ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
69812025-11-13T08:03:03.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69822025-11-13T08:03:03.761ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69832025-11-13T08:03:03.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69842025-11-13T08:03:03.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69852025-11-13T08:03:03.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69862025-11-13T08:03:03.764ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
69872025-11-13T08:03:03.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
69882025-11-13T08:03:03.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
69892025-11-13T08:03:03.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69902025-11-13T08:03:03.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69912025-11-13T08:03:03.768ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
69922025-11-13T08:03:03.768ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
69932025-11-13T08:03:03.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69942025-11-13T08:03:03.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
69952025-11-13T08:03:03.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69962025-11-13T08:03:03.771ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.128529745s iteration = 9 status = {}
69972025-11-13T08:03:03.772ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
69982025-11-13T08:03:03.772ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
69992025-11-13T08:03:03.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70002025-11-13T08:03:03.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70012025-11-13T08:03:03.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70022025-11-13T08:03:03.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70032025-11-13T08:03:03.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70042025-11-13T08:03:03.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70052025-11-13T08:03:03.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
70062025-11-13T08:03:03.782ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
70072025-11-13T08:03:03.782ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
70082025-11-13T08:03:03.782ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = OmicronZones step id = Cleanup
70092025-11-13T08:03:03.782ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = OmicronZones step id = Cleanup
70102025-11-13T08:03:03.782ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = PhysicalDisks step id = Cleanup
70112025-11-13T08:03:03.782ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = Clickhouse step id = Ensure
70122025-11-13T08:03:03.782ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = Clickhouse step id = Ensure
70132025-11-13T08:03:03.812ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
70142025-11-13T08:03:03.817ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "ae1dc462-4663-46b2-9b77-5b7309ea435b", "date": "Thu, 13 Nov 2025 08:03:03 GMT"} })
70152025-11-13T08:03:03.818ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = SupportBundles step id = Cleanup
70162025-11-13T08:03:03.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70172025-11-13T08:03:03.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70182025-11-13T08:03:03.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70192025-11-13T08:03:03.821ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
70202025-11-13T08:03:03.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70212025-11-13T08:03:03.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70222025-11-13T08:03:03.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70232025-11-13T08:03:03.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70242025-11-13T08:03:03.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70252025-11-13T08:03:03.827ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
70262025-11-13T08:03:03.872ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = OmicronZones step id = Cleanup
70272025-11-13T08:03:03.872ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
70282025-11-13T08:03:03.872ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
70292025-11-13T08:03:03.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70302025-11-13T08:03:03.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70312025-11-13T08:03:03.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70322025-11-13T08:03:03.876ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
70332025-11-13T08:03:03.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70342025-11-13T08:03:03.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70352025-11-13T08:03:03.912ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
70362025-11-13T08:03:03.912ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
70372025-11-13T08:03:03.912ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = Cockroach step id = Ensure
70382025-11-13T08:03:03.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
70392025-11-13T08:03:03.912ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
70402025-11-13T08:03:03.930ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
70412025-11-13T08:03:03.930ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 02c3e02b-9961-416a-9207-4038230ff5b0 step component = MgsUpdates step id = Ensure
70422025-11-13T08:03:03.930ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Dependency
70432025-11-13T08:03:03.931ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 14 reason = Dependency
70442025-11-13T08:03:03.931ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70452025-11-13T08:03:03.931ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70462025-11-13T08:03:03.931ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.323379906s iteration = 6 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3290,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":133508865,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":133508815,"secs":0}},"event_index":1,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":133523815,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":253331831,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":253331781,"secs":0}},"event_index":2,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":386880766,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":163183088,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":163183038,"secs":0}},"event_index":3,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":550093044,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":244445554,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":244445514,"secs":0}},"event_index":4,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":794553489,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":247059297,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":247059247,"secs":0}},"event_index":5,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":41628757,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":133138078,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":133138038,"secs":0}},"event_index":6,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":174776195,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4010,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3980,"secs":0}},"event_index":7,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":174792125,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2380,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2340,"secs":0}},"event_index":8,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":174796825,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3500,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3460,"secs":0}},"event_index":9,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":174802295,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1980,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1940,"secs":0}},"event_index":10,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":174806035,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35710572,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":35710512,"secs":0}},"event_index":11,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":210518117,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":54459230,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":54459190,"secs":0}},"event_index":12,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":264989308,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39193181,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":39193141,"secs":0}},"event_index":13,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":304200599,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":18802129,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":18802079,"secs":0}},"event_index":14,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":323021039,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5950,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5910,"secs":0}},"event_index":15,"execution_id":"17ae63d8-3d9b-451c-b763-a48b1d4a9995","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":323035679,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
70472025-11-13T08:03:03.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
70482025-11-13T08:03:03.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70492025-11-13T08:03:03.932ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70502025-11-13T08:03:03.933ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70512025-11-13T08:03:03.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
70522025-11-13T08:03:03.935ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70532025-11-13T08:03:03.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70542025-11-13T08:03:03.935ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
70552025-11-13T08:03:03.935ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70562025-11-13T08:03:03.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
70572025-11-13T08:03:03.936ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70582025-11-13T08:03:03.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70592025-11-13T08:03:03.938ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70602025-11-13T08:03:03.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
70612025-11-13T08:03:03.940ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
70622025-11-13T08:03:03.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
70632025-11-13T08:03:03.942ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
70642025-11-13T08:03:03.942ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 11.509626ms iteration = 14 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
70652025-11-13T08:03:04.069ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(55b45047-b092-4620-af64-6b27b076a7c5, 0), (08c59e42-a65a-4a78-b24e-a61ef72ba1c2, 0), (25faec09-1a95-4211-b5bb-8965f08ec3c6, 0), (036cf7b1-53e4-423a-bec9-1e7a1c588e6a, 0)] collection_id = 55b45047-b092-4620-af64-6b27b076a7c5
70662025-11-13T08:03:04.069ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70672025-11-13T08:03:04.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
70682025-11-13T08:03:04.217ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 55b45047-b092-4620-af64-6b27b076a7c5 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
70692025-11-13T08:03:04.217ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70702025-11-13T08:03:04.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
70712025-11-13T08:03:04.222ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(08c59e42-a65a-4a78-b24e-a61ef72ba1c2, 0), (25faec09-1a95-4211-b5bb-8965f08ec3c6, 0), (036cf7b1-53e4-423a-bec9-1e7a1c588e6a, 0)]
70722025-11-13T08:03:04.222ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6192, tv_nsec: 574781190 } })
70732025-11-13T08:03:04.253ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6192, tv_nsec: 605738119 } })
70742025-11-13T08:03:04.254ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
70752025-11-13T08:03:04.254ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
70762025-11-13T08:03:04.254ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
70772025-11-13T08:03:04.254ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8be948e7-1c97-4de1-a128-4d3fd5f6a522", "content-length": "476", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70782025-11-13T08:03:04.254ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
70792025-11-13T08:03:04.254ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93d1e65e-c9a8-4d1d-bd08-4497c3ce8e1b", "content-length": "734", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70802025-11-13T08:03:04.254ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
70812025-11-13T08:03:04.255ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7700f6fc-6f29-471e-858f-910557d13361", "content-length": "111", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70822025-11-13T08:03:04.255ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
70832025-11-13T08:03:04.255ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa119076-a6f2-4bdb-b781-5b3cc9f6d0b5", "content-length": "111", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70842025-11-13T08:03:04.255ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
70852025-11-13T08:03:04.255ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce2157fb-ba81-4bdb-be43-bfca8dbc5494", "content-length": "163", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70862025-11-13T08:03:04.255ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
70872025-11-13T08:03:04.256ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "25efda54-8169-46c1-b866-9c1f39682a7c", "content-length": "163", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70882025-11-13T08:03:04.256ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
70892025-11-13T08:03:04.256ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e486d96-6a75-4765-b0d8-231a3d66f8ee", "content-length": "166", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70902025-11-13T08:03:04.256ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
70912025-11-13T08:03:04.256ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6239e45a-bc5a-4c3d-869c-cb476182ff21", "content-length": "166", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70922025-11-13T08:03:04.256ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
70932025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e7f06c0-603d-45ac-bd0d-86f41816ddde", "content-length": "702", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70942025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
70952025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2166aa90-84aa-4375-8eb5-1a4304b205d8", "content-length": "727", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70962025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
70972025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "68ac6f85-790e-47ee-a7cd-7771602df23b", "content-length": "729", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
70982025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
70992025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d8de519-4e79-4e9e-beff-2cc3de829057", "content-length": "728", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71002025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
71012025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6798f11-ca08-4bb3-88d6-46efd90c9dea", "content-length": "722", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71022025-11-13T08:03:04.257ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
71032025-11-13T08:03:04.258ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a6e279e-fd62-4ff4-bf44-dfa2c4627905", "content-length": "10", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71042025-11-13T08:03:04.258ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
71052025-11-13T08:03:04.258ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f0ca0b7-3fe8-4e83-966c-c45ac458a01f", "content-length": "144", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71062025-11-13T08:03:04.258ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
71072025-11-13T08:03:04.258ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "921062c6-12cc-4bd9-bc1e-8c543a313859", "content-length": "147", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71082025-11-13T08:03:04.258ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
71092025-11-13T08:03:04.259ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d32e84e-61f5-4c1f-9572-356b5db60993", "content-length": "109", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71102025-11-13T08:03:04.259ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
71112025-11-13T08:03:04.259ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd3100fe-04b0-43be-b2c7-bfe7d13e4ca5", "content-length": "109", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71122025-11-13T08:03:04.259ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
71132025-11-13T08:03:04.260ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd9cf3de-e1d4-4c24-91f5-750986319da3", "content-length": "163", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71142025-11-13T08:03:04.260ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
71152025-11-13T08:03:04.260ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa9da058-3254-44ee-afbe-9a4f4545a3ef", "content-length": "163", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71162025-11-13T08:03:04.260ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
71172025-11-13T08:03:04.261ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "77b0df29-09f0-4fa1-ac1e-d4e5954aec4c", "content-length": "166", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71182025-11-13T08:03:04.261ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
71192025-11-13T08:03:04.261ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "066f27f3-dfa0-4204-b71d-726839b72ea8", "content-length": "166", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71202025-11-13T08:03:04.261ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
71212025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b87b66d5-5cc1-4bfa-b18c-e02c7db12951", "content-length": "702", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71222025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
71232025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "87d13e89-bf2a-401a-ad11-5aeefc0e4873", "content-length": "727", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71242025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
71252025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27fab6e-52e7-48dc-ad09-2567d2bdb937", "content-length": "729", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71262025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
71272025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "966f470d-d42a-484c-bc49-da21d4350105", "content-length": "728", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71282025-11-13T08:03:04.262ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71292025-11-13T08:03:04.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71302025-11-13T08:03:04.265ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71312025-11-13T08:03:04.270ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
71322025-11-13T08:03:04.270ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71332025-11-13T08:03:04.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71342025-11-13T08:03:04.272ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71352025-11-13T08:03:04.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
71362025-11-13T08:03:04.275ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71372025-11-13T08:03:04.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71382025-11-13T08:03:04.308ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
71392025-11-13T08:03:04.308ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
71402025-11-13T08:03:04.391ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "789d6f55-c21a-415b-b456-bcdfd499fa10", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71412025-11-13T08:03:04.393ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
71422025-11-13T08:03:04.393ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
71432025-11-13T08:03:04.393ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
71442025-11-13T08:03:04.393ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
71452025-11-13T08:03:04.393ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
71462025-11-13T08:03:04.393ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
71472025-11-13T08:03:04.393ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
71482025-11-13T08:03:04.393ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
71492025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a254aae5-7f1c-45b1-a56d-da3eaed8dd8b", "content-length": "64", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71502025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d4377a6-defd-4c5d-a4b0-597663b39d17", "content-length": "64", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71512025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
71522025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8456bf0b-26d7-4fbc-b459-f90244a1f4de", "content-length": "64", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71532025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
71542025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
71552025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "af600ea3-2b5f-4deb-8521-e96f4f934c5e", "content-length": "64", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71562025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "42520955-ca42-45cc-a8ec-72f91d131455", "content-length": "64", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71572025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
71582025-11-13T08:03:04.394ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
71592025-11-13T08:03:04.434ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "84eaf5d1-8178-422f-9f63-0d2bfea084fa", "content-length": "387260", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71602025-11-13T08:03:04.435ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "57efa31e-5786-4c1d-bb45-fa1ab05fc8ed", "content-length": "389101", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71612025-11-13T08:03:04.435ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6de39d17-5a8c-4c96-96f0-810107d98d5f", "content-length": "384349", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71622025-11-13T08:03:04.442ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "17724e30-b7e9-4ce7-bca1-e46f273a9481", "content-length": "413705", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71632025-11-13T08:03:04.442ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "353df0b1-1029-41a9-abdc-52538133ae51", "content-length": "395607", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71642025-11-13T08:03:04.445ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
71652025-11-13T08:03:04.445ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
71662025-11-13T08:03:04.445ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
71672025-11-13T08:03:04.445ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
71682025-11-13T08:03:04.445ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
71692025-11-13T08:03:04.475ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
71702025-11-13T08:03:04.475ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
71712025-11-13T08:03:04.479ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "76778d45-1fd9-4866-9b40-2c57a2581d72", "content-length": "120", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71722025-11-13T08:03:04.479ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
71732025-11-13T08:03:04.571ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
71742025-11-13T08:03:04.571ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
71752025-11-13T08:03:04.572ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b389005-b05d-476d-aa6c-eeb1a77c420a", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71762025-11-13T08:03:04.572ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
71772025-11-13T08:03:04.574ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
71782025-11-13T08:03:04.574ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
71792025-11-13T08:03:04.574ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbef53d1-c9ca-4ebe-ad03-c029f8ea254d", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71802025-11-13T08:03:04.574ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
71812025-11-13T08:03:04.575ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
71822025-11-13T08:03:04.575ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
71832025-11-13T08:03:04.576ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffa7d6ea-ae77-4c21-8273-be86b4f021f3", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:03:04 GMT"} })
71842025-11-13T08:03:04.576ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
71852025-11-13T08:03:04.578ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
71862025-11-13T08:03:04.578ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
71872025-11-13T08:03:04.580ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71882025-11-13T08:03:04.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71892025-11-13T08:03:04.582ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71902025-11-13T08:03:04.584ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
71912025-11-13T08:03:04.584ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71922025-11-13T08:03:04.584ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
71932025-11-13T08:03:04.788ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
71942025-11-13T08:03:04.788ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7 time_started = 2025-11-13 08:03:04.254043 UTC
71952025-11-13T08:03:04.788ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 23 reason = Dependency
71962025-11-13T08:03:04.788ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
71972025-11-13T08:03:04.788ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 857.845444ms iteration = 6 status = {"collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","time_done":"2025-11-13 08:03:04.578261 UTC","time_started":"2025-11-13 08:03:04.254043 UTC"}
71982025-11-13T08:03:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
71992025-11-13T08:03:04.790ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72002025-11-13T08:03:04.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
72012025-11-13T08:03:04.793ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
72022025-11-13T08:03:04.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
72032025-11-13T08:03:04.796ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
72042025-11-13T08:03:04.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
72052025-11-13T08:03:04.843ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 14 reason = Dependency
72062025-11-13T08:03:04.843ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 55.239136ms iteration = 23 status = {"Loaded":{"collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","time_loaded":"2025-11-13T08:03:04.788683066Z","time_started":"2025-11-13T08:03:04.254043Z"}}
72072025-11-13T08:03:04.843ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 8 reason = Dependency
72082025-11-13T08:03:04.843ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 16 reason = Dependency
72092025-11-13T08:03:04.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72102025-11-13T08:03:04.844ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72112025-11-13T08:03:04.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72122025-11-13T08:03:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72132025-11-13T08:03:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72142025-11-13T08:03:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72152025-11-13T08:03:04.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72162025-11-13T08:03:04.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72172025-11-13T08:03:04.846ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72182025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72192025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72202025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72212025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72222025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72232025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72242025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72252025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72262025-11-13T08:03:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72272025-11-13T08:03:04.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72282025-11-13T08:03:04.851ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72292025-11-13T08:03:04.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72302025-11-13T08:03:04.852ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72312025-11-13T08:03:04.853ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72322025-11-13T08:03:04.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72332025-11-13T08:03:04.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72342025-11-13T08:03:04.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72352025-11-13T08:03:04.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72362025-11-13T08:03:04.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72372025-11-13T08:03:04.855ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72382025-11-13T08:03:04.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72392025-11-13T08:03:04.856ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
72402025-11-13T08:03:04.856ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
72412025-11-13T08:03:04.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72422025-11-13T08:03:04.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
72432025-11-13T08:03:04.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72442025-11-13T08:03:04.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72452025-11-13T08:03:04.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72462025-11-13T08:03:04.859ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72472025-11-13T08:03:04.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72482025-11-13T08:03:04.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72492025-11-13T08:03:04.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72502025-11-13T08:03:04.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72512025-11-13T08:03:04.862ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72522025-11-13T08:03:04.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72532025-11-13T08:03:04.862ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72542025-11-13T08:03:04.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72552025-11-13T08:03:04.864ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72562025-11-13T08:03:04.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72572025-11-13T08:03:04.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72582025-11-13T08:03:04.866ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
72592025-11-13T08:03:04.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72602025-11-13T08:03:04.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72612025-11-13T08:03:04.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
72622025-11-13T08:03:04.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72632025-11-13T08:03:04.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72642025-11-13T08:03:04.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72652025-11-13T08:03:04.870ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72662025-11-13T08:03:04.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72672025-11-13T08:03:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72682025-11-13T08:03:04.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72692025-11-13T08:03:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72702025-11-13T08:03:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72712025-11-13T08:03:04.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72722025-11-13T08:03:04.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72732025-11-13T08:03:04.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72742025-11-13T08:03:04.874ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
72752025-11-13T08:03:04.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
72762025-11-13T08:03:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
72772025-11-13T08:03:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
72782025-11-13T08:03:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72792025-11-13T08:03:04.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72802025-11-13T08:03:04.881ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
72812025-11-13T08:03:04.881ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 37.841394ms iteration = 8 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
72822025-11-13T08:03:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72832025-11-13T08:03:04.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72842025-11-13T08:03:04.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72852025-11-13T08:03:04.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
72862025-11-13T08:03:04.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72872025-11-13T08:03:04.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72882025-11-13T08:03:04.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72892025-11-13T08:03:04.896ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
72902025-11-13T08:03:04.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72912025-11-13T08:03:04.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72922025-11-13T08:03:04.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72932025-11-13T08:03:04.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72942025-11-13T08:03:04.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72952025-11-13T08:03:04.901ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 57.742635ms iteration = 16 status = {"physical_disks_added":0}
72962025-11-13T08:03:04.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
72972025-11-13T08:03:04.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
72982025-11-13T08:03:04.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
72992025-11-13T08:03:04.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73002025-11-13T08:03:04.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73012025-11-13T08:03:04.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73022025-11-13T08:03:04.909ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
73032025-11-13T08:03:04.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73042025-11-13T08:03:04.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73052025-11-13T08:03:04.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73062025-11-13T08:03:04.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73072025-11-13T08:03:04.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73082025-11-13T08:03:04.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
73092025-11-13T08:03:04.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73102025-11-13T08:03:04.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73112025-11-13T08:03:04.928ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73122025-11-13T08:03:04.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73132025-11-13T08:03:04.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73142025-11-13T08:03:04.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73152025-11-13T08:03:04.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73162025-11-13T08:03:04.934ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73172025-11-13T08:03:04.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73182025-11-13T08:03:04.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73192025-11-13T08:03:04.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73202025-11-13T08:03:04.939ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73212025-11-13T08:03:04.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73222025-11-13T08:03:04.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73232025-11-13T08:03:04.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73242025-11-13T08:03:04.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73252025-11-13T08:03:04.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73262025-11-13T08:03:04.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73272025-11-13T08:03:04.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73282025-11-13T08:03:04.948ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73292025-11-13T08:03:04.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73302025-11-13T08:03:04.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73312025-11-13T08:03:04.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73322025-11-13T08:03:04.952ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
73332025-11-13T08:03:04.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73342025-11-13T08:03:04.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73352025-11-13T08:03:04.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73362025-11-13T08:03:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73372025-11-13T08:03:04.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73382025-11-13T08:03:04.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73392025-11-13T08:03:04.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73402025-11-13T08:03:04.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
73412025-11-13T08:03:04.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73422025-11-13T08:03:04.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73432025-11-13T08:03:04.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73442025-11-13T08:03:04.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73452025-11-13T08:03:04.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73462025-11-13T08:03:04.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
73472025-11-13T08:03:04.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73482025-11-13T08:03:04.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73492025-11-13T08:03:04.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73502025-11-13T08:03:04.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73512025-11-13T08:03:04.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73522025-11-13T08:03:04.975ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
73532025-11-13T08:03:04.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73542025-11-13T08:03:04.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73552025-11-13T08:03:04.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73562025-11-13T08:03:04.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73572025-11-13T08:03:04.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73582025-11-13T08:03:04.982ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
73592025-11-13T08:03:04.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73602025-11-13T08:03:04.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73612025-11-13T08:03:04.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73622025-11-13T08:03:04.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73632025-11-13T08:03:04.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73642025-11-13T08:03:04.989ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
73652025-11-13T08:03:04.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73662025-11-13T08:03:04.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73672025-11-13T08:03:04.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73682025-11-13T08:03:04.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73692025-11-13T08:03:04.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73702025-11-13T08:03:04.995ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
73712025-11-13T08:03:04.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73722025-11-13T08:03:04.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73732025-11-13T08:03:05.031ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
73742025-11-13T08:03:05.031ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
73752025-11-13T08:03:05.031ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 3f88d62d-9ab4-4f50-8d57-a0a0d5a1f4b4 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
73762025-11-13T08:03:05.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73772025-11-13T08:03:05.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73782025-11-13T08:03:05.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73792025-11-13T08:03:05.036ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
73802025-11-13T08:03:05.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
73812025-11-13T08:03:05.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
73822025-11-13T08:03:05.038ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
73832025-11-13T08:03:05.038ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
73842025-11-13T08:03:05.038ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 194.872851ms iteration = 14 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
73852025-11-13T08:03:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 422 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 04990d76-d8de-4747-b4bd-5227374b94f1 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
73862025-11-13T08:03:07.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 50 reason = Timeout
73872025-11-13T08:03:07.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73882025-11-13T08:03:07.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73892025-11-13T08:03:07.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73902025-11-13T08:03:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
73912025-11-13T08:03:07.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73922025-11-13T08:03:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
73932025-11-13T08:03:07.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
73942025-11-13T08:03:07.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.722742ms iteration = 50 status = {"config_updated":false}
73952025-11-13T08:03:12.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 26 reason = Timeout
73962025-11-13T08:03:12.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
73972025-11-13T08:03:12.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 51 reason = Timeout
73982025-11-13T08:03:12.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
73992025-11-13T08:03:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74002025-11-13T08:03:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74012025-11-13T08:03:12.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74022025-11-13T08:03:12.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74032025-11-13T08:03:12.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74042025-11-13T08:03:12.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74052025-11-13T08:03:12.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74062025-11-13T08:03:12.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74072025-11-13T08:03:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74082025-11-13T08:03:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74092025-11-13T08:03:12.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74102025-11-13T08:03:12.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74112025-11-13T08:03:12.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
74122025-11-13T08:03:12.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.272343ms iteration = 51 status = {"config_updated":false}
74132025-11-13T08:03:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74142025-11-13T08:03:12.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
74152025-11-13T08:03:12.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74162025-11-13T08:03:12.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74172025-11-13T08:03:12.640ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
74182025-11-13T08:03:12.640ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.120174ms iteration = 26 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
74192025-11-13T08:03:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 26 reason = Timeout
74202025-11-13T08:03:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74212025-11-13T08:03:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74222025-11-13T08:03:12.684ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74232025-11-13T08:03:12.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74242025-11-13T08:03:12.726ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 83.777877ms iteration = 26 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74252025-11-13T08:03:15.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 312 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 8a226d8a-39d1-4b31-9310-ad7a425b1541 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
74262025-11-13T08:03:17.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 24 reason = Timeout
74272025-11-13T08:03:17.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 52 reason = Timeout
74282025-11-13T08:03:17.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
74292025-11-13T08:03:17.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74302025-11-13T08:03:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
74312025-11-13T08:03:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74322025-11-13T08:03:17.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74332025-11-13T08:03:17.620ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74342025-11-13T08:03:17.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74352025-11-13T08:03:17.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74362025-11-13T08:03:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74372025-11-13T08:03:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
74382025-11-13T08:03:17.624ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
74392025-11-13T08:03:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
74402025-11-13T08:03:17.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
74412025-11-13T08:03:17.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.273753ms iteration = 52 status = {"config_updated":false}
74422025-11-13T08:03:17.625ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7 original_time_started = 2025-11-13 08:03:04.254043 UTC
74432025-11-13T08:03:17.625ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.606819ms iteration = 24 status = {"Loaded":{"collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","time_loaded":"2025-11-13T08:03:17.618108163Z","time_started":"2025-11-13T08:03:04.254043Z"}}
74442025-11-13T08:03:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 18 reason = Timeout
74452025-11-13T08:03:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 18 reason = Timeout
74462025-11-13T08:03:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
74472025-11-13T08:03:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74482025-11-13T08:03:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74492025-11-13T08:03:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
74502025-11-13T08:03:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74512025-11-13T08:03:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74522025-11-13T08:03:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74532025-11-13T08:03:17.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74542025-11-13T08:03:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
74552025-11-13T08:03:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
74562025-11-13T08:03:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
74572025-11-13T08:03:17.653ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
74582025-11-13T08:03:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.348085ms iteration = 18 status = "NoSitrep"
74592025-11-13T08:03:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74602025-11-13T08:03:17.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74612025-11-13T08:03:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
74622025-11-13T08:03:17.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
74632025-11-13T08:03:17.658ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74642025-11-13T08:03:17.659ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
74652025-11-13T08:03:17.660ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
74662025-11-13T08:03:17.660ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.541117ms iteration = 18 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
74672025-11-13T08:03:22.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 27 reason = Timeout
74682025-11-13T08:03:22.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 53 reason = Timeout
74692025-11-13T08:03:22.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74702025-11-13T08:03:22.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74712025-11-13T08:03:22.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74722025-11-13T08:03:22.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74732025-11-13T08:03:22.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74742025-11-13T08:03:22.621ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74752025-11-13T08:03:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
74762025-11-13T08:03:22.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74772025-11-13T08:03:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74782025-11-13T08:03:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
74792025-11-13T08:03:22.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
74802025-11-13T08:03:22.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
74812025-11-13T08:03:22.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74822025-11-13T08:03:22.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74832025-11-13T08:03:22.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
74842025-11-13T08:03:22.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.940557ms iteration = 53 status = {"config_updated":false}
74852025-11-13T08:03:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74862025-11-13T08:03:22.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
74872025-11-13T08:03:22.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
74882025-11-13T08:03:22.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
74892025-11-13T08:03:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 27 reason = Timeout
74902025-11-13T08:03:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74912025-11-13T08:03:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74922025-11-13T08:03:22.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
74932025-11-13T08:03:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
74942025-11-13T08:03:22.644ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
74952025-11-13T08:03:22.644ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.864508ms iteration = 27 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
74962025-11-13T08:03:22.677ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.670621ms iteration = 27 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
74972025-11-13T08:03:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 274 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 0143603c-b6a6-4c6a-a298-4b0d0de482b8 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
74982025-11-13T08:03:27.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 54 reason = Timeout
74992025-11-13T08:03:27.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75002025-11-13T08:03:27.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75012025-11-13T08:03:27.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75022025-11-13T08:03:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
75032025-11-13T08:03:27.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75042025-11-13T08:03:27.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75052025-11-13T08:03:27.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
75062025-11-13T08:03:27.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.705521ms iteration = 54 status = {"config_updated":false}
75072025-11-13T08:03:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 10 reason = Timeout
75082025-11-13T08:03:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 10 reason = Timeout
75092025-11-13T08:03:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
75102025-11-13T08:03:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 10 reason = Timeout
75112025-11-13T08:03:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
75122025-11-13T08:03:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
75132025-11-13T08:03:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
75142025-11-13T08:03:32.612ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6220, tv_nsec: 965161404 } })
75152025-11-13T08:03:32.613ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
75162025-11-13T08:03:32.614ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6220, tv_nsec: 966798216 } })
75172025-11-13T08:03:32.615ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.14969ms iteration = 10 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
75182025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 15 reason = Timeout
75192025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 25 reason = Timeout
75202025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 28 reason = Timeout
75212025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
75222025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 17 reason = Timeout
75232025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75242025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
75252025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75262025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 55 reason = Timeout
75272025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75282025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75292025-11-13T08:03:32.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75302025-11-13T08:03:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75312025-11-13T08:03:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
75322025-11-13T08:03:32.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75332025-11-13T08:03:32.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75342025-11-13T08:03:32.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75352025-11-13T08:03:32.621ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75362025-11-13T08:03:32.621ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75372025-11-13T08:03:32.621ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75382025-11-13T08:03:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
75392025-11-13T08:03:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75402025-11-13T08:03:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75412025-11-13T08:03:32.627ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75422025-11-13T08:03:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75432025-11-13T08:03:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75442025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
75452025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
75462025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
75472025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
75482025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
75492025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
75502025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
75512025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
75522025-11-13T08:03:32.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
75532025-11-13T08:03:32.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75542025-11-13T08:03:32.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75552025-11-13T08:03:32.631ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
75562025-11-13T08:03:32.631ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 12.841732ms iteration = 55 status = {"config_updated":false}
75572025-11-13T08:03:32.631ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7 original_time_started = 2025-11-13 08:03:04.254043 UTC
75582025-11-13T08:03:32.631ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.415564ms iteration = 25 status = {"Loaded":{"collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","time_loaded":"2025-11-13T08:03:32.618380259Z","time_started":"2025-11-13T08:03:04.254043Z"}}
75592025-11-13T08:03:32.632ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75602025-11-13T08:03:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75612025-11-13T08:03:32.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75622025-11-13T08:03:32.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75632025-11-13T08:03:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
75642025-11-13T08:03:32.638ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
75652025-11-13T08:03:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
75662025-11-13T08:03:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
75672025-11-13T08:03:32.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
75682025-11-13T08:03:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
75692025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 10 reason = Timeout
75702025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 15 reason = Timeout
75712025-11-13T08:03:32.641ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
75722025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 10 reason = Timeout
75732025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
75742025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
75752025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 10 reason = Timeout
75762025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
75772025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
75782025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
75792025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 28 reason = Timeout
75802025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
75812025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 10 reason = Timeout
75822025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
75832025-11-13T08:03:32.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 10 reason = Timeout
75842025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 10 reason = Timeout
75852025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
75862025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
75872025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
75882025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
75892025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
75902025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
75912025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
75922025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
75932025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
75942025-11-13T08:03:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
75952025-11-13T08:03:32.646ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75962025-11-13T08:03:32.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75972025-11-13T08:03:32.646ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75982025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 10 reason = Timeout
75992025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 10 reason = Timeout
76002025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76012025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
76022025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 19 reason = Timeout
76032025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 19 reason = Timeout
76042025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
76052025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76062025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 10 reason = Timeout
76072025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
76082025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 10 reason = Timeout
76092025-11-13T08:03:32.647ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76102025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 10 reason = Timeout
76112025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 10 reason = Timeout
76122025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
76132025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76142025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
76152025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76162025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
76172025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
76182025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76192025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
76202025-11-13T08:03:32.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6221, tv_nsec: 997049 } })
76212025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
76222025-11-13T08:03:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76232025-11-13T08:03:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76242025-11-13T08:03:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76252025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76262025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76272025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
76282025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
76292025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76302025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
76312025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
76322025-11-13T08:03:32.651ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
76332025-11-13T08:03:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
76342025-11-13T08:03:32.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76352025-11-13T08:03:32.656ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76362025-11-13T08:03:32.658ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
76372025-11-13T08:03:32.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
76382025-11-13T08:03:32.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
76392025-11-13T08:03:32.659ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
76402025-11-13T08:03:32.659ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6221, tv_nsec: 11762000 } })
76412025-11-13T08:03:32.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
76422025-11-13T08:03:32.659ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
76432025-11-13T08:03:32.660ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
76442025-11-13T08:03:32.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
76452025-11-13T08:03:32.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
76462025-11-13T08:03:32.660ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 19.004853ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
76472025-11-13T08:03:32.666ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 18.367241ms iteration = 19 status = "NoSitrep"
76482025-11-13T08:03:32.687ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
76492025-11-13T08:03:32.687ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 68.874184ms iteration = 28 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
76502025-11-13T08:03:32.706ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76512025-11-13T08:03:32.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76522025-11-13T08:03:32.711ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76532025-11-13T08:03:32.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
76542025-11-13T08:03:32.714ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
76552025-11-13T08:03:32.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
76562025-11-13T08:03:32.716ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
76572025-11-13T08:03:32.716ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 97.953586ms iteration = 15 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
76582025-11-13T08:03:32.717ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76592025-11-13T08:03:32.719ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 71.617398ms iteration = 10 status = {"errors":[],"finish_invoked_ok":[]}
76602025-11-13T08:03:32.720ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76612025-11-13T08:03:32.720ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
76622025-11-13T08:03:32.720ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76632025-11-13T08:03:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
76642025-11-13T08:03:32.721ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
76652025-11-13T08:03:32.721ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
76662025-11-13T08:03:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
76672025-11-13T08:03:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
76682025-11-13T08:03:32.722ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
76692025-11-13T08:03:32.722ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
76702025-11-13T08:03:32.722ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
76712025-11-13T08:03:32.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
76722025-11-13T08:03:32.723ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
76732025-11-13T08:03:32.723ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
76742025-11-13T08:03:32.723ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
76752025-11-13T08:03:32.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
76762025-11-13T08:03:32.724ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 76.314391ms iteration = 10 status = {"errors":[],"garbage_collect_requested":[]}
76772025-11-13T08:03:32.724ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 106.079226ms iteration = 17 status = {"physical_disks_added":0}
76782025-11-13T08:03:32.724ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76792025-11-13T08:03:32.725ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 118.086142ms iteration = 10 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
76802025-11-13T08:03:32.725ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 83.605323ms iteration = 10 status = 0
76812025-11-13T08:03:32.726ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
76822025-11-13T08:03:32.726ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 78.798469ms iteration = 10 status = {"errors":[],"requests_created_ok":[]}
76832025-11-13T08:03:32.727ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76842025-11-13T08:03:32.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
76852025-11-13T08:03:32.728ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
76862025-11-13T08:03:32.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
76872025-11-13T08:03:32.729ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76882025-11-13T08:03:32.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76892025-11-13T08:03:32.730ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
76902025-11-13T08:03:32.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
76912025-11-13T08:03:32.732ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76922025-11-13T08:03:32.734ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
76932025-11-13T08:03:32.735ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
76942025-11-13T08:03:32.735ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 87.465849ms iteration = 19 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
76952025-11-13T08:03:32.737ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
76962025-11-13T08:03:32.737ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 95.216122ms iteration = 10 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
76972025-11-13T08:03:32.738ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
76982025-11-13T08:03:32.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
76992025-11-13T08:03:32.739ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77002025-11-13T08:03:32.742ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
77012025-11-13T08:03:32.742ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77022025-11-13T08:03:32.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
77032025-11-13T08:03:32.743ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77042025-11-13T08:03:32.745ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
77052025-11-13T08:03:32.745ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
77062025-11-13T08:03:32.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
77072025-11-13T08:03:32.764ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 122.128111ms iteration = 28 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
77082025-11-13T08:03:32.784ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
77092025-11-13T08:03:32.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
77102025-11-13T08:03:32.786ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
77112025-11-13T08:03:32.786ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
77122025-11-13T08:03:32.786ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 144.333158ms iteration = 10 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
77132025-11-13T08:03:32.792ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 144.227466ms iteration = 10 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
77142025-11-13T08:03:32.830ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 183.020568ms iteration = 10 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
77152025-11-13T08:03:32.834ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
77162025-11-13T08:03:32.834ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77172025-11-13T08:03:32.834ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c17ddc8-29c1-4a7f-96e6-181f0c05b4b9", "content-length": "26", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77182025-11-13T08:03:32.834ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77192025-11-13T08:03:32.835ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
77202025-11-13T08:03:32.904ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
77212025-11-13T08:03:32.904ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
77222025-11-13T08:03:32.904ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "587829f1-01df-4868-9a71-d7611f65c29d", "content-length": "26", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77232025-11-13T08:03:32.904ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
77242025-11-13T08:03:32.905ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
77252025-11-13T08:03:32.926ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
77262025-11-13T08:03:32.927ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb8ecfec-c03a-4bdf-bd27-9b827aed28c7", "content-length": "78", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77272025-11-13T08:03:32.927ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
77282025-11-13T08:03:32.927ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
77292025-11-13T08:03:32.927ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
77302025-11-13T08:03:32.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
77312025-11-13T08:03:32.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
77322025-11-13T08:03:32.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
77332025-11-13T08:03:32.928ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
77342025-11-13T08:03:32.929ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
77352025-11-13T08:03:32.930ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
77362025-11-13T08:03:32.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
77372025-11-13T08:03:32.932ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
77382025-11-13T08:03:32.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
77392025-11-13T08:03:32.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
77402025-11-13T08:03:32.933ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
77412025-11-13T08:03:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
77422025-11-13T08:03:32.933ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
77432025-11-13T08:03:32.933ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
77442025-11-13T08:03:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
77452025-11-13T08:03:32.935ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
77462025-11-13T08:03:32.935ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
77472025-11-13T08:03:32.935ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "153c0860-23c6-4cb9-ad26-ccbd40a5cccc", "content-length": "81", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77482025-11-13T08:03:32.971ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
77492025-11-13T08:03:32.971ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
77502025-11-13T08:03:32.972ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
77512025-11-13T08:03:32.972ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
77522025-11-13T08:03:32.972ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "53cd3b53-141f-4849-8b1a-60f588d34a1e", "content-length": "81", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77532025-11-13T08:03:32.972ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "43c94665-0ca7-4881-9b90-786795a0447e", "content-length": "81", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77542025-11-13T08:03:32.974ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
77552025-11-13T08:03:32.974ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
77562025-11-13T08:03:32.974ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77572025-11-13T08:03:32.974ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 326.362486ms iteration = 10 status = {"disabled":false,"errors":[],"sps":[]}
77582025-11-13T08:03:32.974ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fcb217f-091c-4f54-92d2-af718ab39127", "content-length": "372", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77592025-11-13T08:03:32.974ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77602025-11-13T08:03:32.974ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "21744808-f77c-49e9-a3a3-a96a5a4c46f1", "content-length": "691", "date": "Thu, 13 Nov 2025 08:03:32 GMT"} })
77612025-11-13T08:03:32.975ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
77622025-11-13T08:03:32.975ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
77632025-11-13T08:03:32.975ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
77642025-11-13T08:03:32.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77652025-11-13T08:03:32.976ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 334.767881ms iteration = 10 status = {}
77662025-11-13T08:03:33.002ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6221, tv_nsec: 355054989 } })
77672025-11-13T08:03:33.004ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 397.25389ms iteration = 10 status = {}
77682025-11-13T08:03:33.028ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
77692025-11-13T08:03:33.028ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
77702025-11-13T08:03:33.031ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "5d285de8-6ee4-4b2e-8ca6-12e32ed02b9d", "date": "Thu, 13 Nov 2025 08:03:33 GMT"} })
77712025-11-13T08:03:33.031ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77722025-11-13T08:03:33.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77732025-11-13T08:03:33.034ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77742025-11-13T08:03:33.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77752025-11-13T08:03:33.038ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
77762025-11-13T08:03:33.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
77772025-11-13T08:03:33.041ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
77782025-11-13T08:03:33.041ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
77792025-11-13T08:03:33.046ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "6a373f76-823c-4a60-aa69-e2c4b42105e5", "date": "Thu, 13 Nov 2025 08:03:33 GMT"} })
77802025-11-13T08:03:33.047ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 405.93157ms iteration = 15 status = {}
77812025-11-13T08:03:33.067ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77822025-11-13T08:03:33.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77832025-11-13T08:03:33.070ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
77842025-11-13T08:03:33.070ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77852025-11-13T08:03:33.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77862025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
77872025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
77882025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
77892025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
77902025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77912025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77922025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77932025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77942025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
77952025-11-13T08:03:33.081ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
77962025-11-13T08:03:33.082ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
77972025-11-13T08:03:33.082ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
77982025-11-13T08:03:33.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
77992025-11-13T08:03:33.084ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
78002025-11-13T08:03:33.084ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
78012025-11-13T08:03:33.261ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ea7db97d-18e4-406f-8a73-fde0781ddc9a", "date": "Thu, 13 Nov 2025 08:03:33 GMT"} })
78022025-11-13T08:03:33.261ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
78032025-11-13T08:03:33.261ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
78042025-11-13T08:03:33.261ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "251c2f52-e02a-464a-ba34-f339a9316aa5", "content-length": "682", "date": "Thu, 13 Nov 2025 08:03:33 GMT"} })
78052025-11-13T08:03:33.261ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78062025-11-13T08:03:33.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78072025-11-13T08:03:33.264ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78082025-11-13T08:03:33.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78092025-11-13T08:03:33.267ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78102025-11-13T08:03:33.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78112025-11-13T08:03:33.269ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
78122025-11-13T08:03:33.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
78132025-11-13T08:03:33.276ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 634.858791ms iteration = 10 status = {}
78142025-11-13T08:03:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 308 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = fe0159a7-8b1a-48fa-aeb5-d34ab1d249a8 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
78152025-11-13T08:03:37.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 56 reason = Timeout
78162025-11-13T08:03:37.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78172025-11-13T08:03:37.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78182025-11-13T08:03:37.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78192025-11-13T08:03:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
78202025-11-13T08:03:37.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78212025-11-13T08:03:37.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78222025-11-13T08:03:37.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
78232025-11-13T08:03:37.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.818315ms iteration = 56 status = {"config_updated":false}
78242025-11-13T08:03:42.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 29 reason = Timeout
78252025-11-13T08:03:42.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 57 reason = Timeout
78262025-11-13T08:03:42.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78272025-11-13T08:03:42.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78282025-11-13T08:03:42.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78292025-11-13T08:03:42.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78302025-11-13T08:03:42.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78312025-11-13T08:03:42.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78322025-11-13T08:03:42.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
78332025-11-13T08:03:42.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78342025-11-13T08:03:42.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78352025-11-13T08:03:42.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
78362025-11-13T08:03:42.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78372025-11-13T08:03:42.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78382025-11-13T08:03:42.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
78392025-11-13T08:03:42.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.11875ms iteration = 57 status = {"config_updated":false}
78402025-11-13T08:03:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78412025-11-13T08:03:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78422025-11-13T08:03:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78432025-11-13T08:03:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
78442025-11-13T08:03:42.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
78452025-11-13T08:03:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
78462025-11-13T08:03:42.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 29 reason = Timeout
78472025-11-13T08:03:42.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78482025-11-13T08:03:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78492025-11-13T08:03:42.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
78502025-11-13T08:03:42.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
78512025-11-13T08:03:42.643ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
78522025-11-13T08:03:42.643ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.115223ms iteration = 29 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
78532025-11-13T08:03:42.686ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 45.090816ms iteration = 29 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
78542025-11-13T08:03:45.473ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:38259
78552025-11-13T08:03:45.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = f6d0f75d-d9df-40a3-8f73-bb8e8ec7d4f8 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
78562025-11-13T08:03:45.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = f6d0f75d-d9df-40a3-8f73-bb8e8ec7d4f8 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
78572025-11-13T08:03:45.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = f6d0f75d-d9df-40a3-8f73-bb8e8ec7d4f8 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
78582025-11-13T08:03:45.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2632 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = f6d0f75d-d9df-40a3-8f73-bb8e8ec7d4f8 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
78592025-11-13T08:03:45.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = 6025e6a2-d83d-4125-9e5d-041a3a3f8374 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
78602025-11-13T08:03:45.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = 6025e6a2-d83d-4125-9e5d-041a3a3f8374 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
78612025-11-13T08:03:45.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = 6025e6a2-d83d-4125-9e5d-041a3a3f8374 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
78622025-11-13T08:03:45.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2479 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:38259 req_id = 6025e6a2-d83d-4125-9e5d-041a3a3f8374 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
78632025-11-13T08:03:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 269 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = c4d1cad9-3883-4428-a0f6-c950b3ce9793 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
78642025-11-13T08:03:47.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 26 reason = Timeout
78652025-11-13T08:03:47.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 58 reason = Timeout
78662025-11-13T08:03:47.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
78672025-11-13T08:03:47.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78682025-11-13T08:03:47.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
78692025-11-13T08:03:47.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78702025-11-13T08:03:47.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78712025-11-13T08:03:47.620ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78722025-11-13T08:03:47.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
78732025-11-13T08:03:47.624ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
78742025-11-13T08:03:47.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
78752025-11-13T08:03:47.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
78762025-11-13T08:03:47.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
78772025-11-13T08:03:47.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
78782025-11-13T08:03:47.626ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7 original_time_started = 2025-11-13 08:03:04.254043 UTC
78792025-11-13T08:03:47.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
78802025-11-13T08:03:47.626ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.194401ms iteration = 26 status = {"Loaded":{"collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","time_loaded":"2025-11-13T08:03:47.618567182Z","time_started":"2025-11-13T08:03:04.254043Z"}}
78812025-11-13T08:03:47.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.175471ms iteration = 58 status = {"config_updated":false}
78822025-11-13T08:03:47.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 20 reason = Timeout
78832025-11-13T08:03:47.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
78842025-11-13T08:03:47.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 20 reason = Timeout
78852025-11-13T08:03:47.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
78862025-11-13T08:03:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
78872025-11-13T08:03:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
78882025-11-13T08:03:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
78892025-11-13T08:03:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
78902025-11-13T08:03:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78912025-11-13T08:03:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78922025-11-13T08:03:47.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
78932025-11-13T08:03:47.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
78942025-11-13T08:03:47.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
78952025-11-13T08:03:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
78962025-11-13T08:03:47.652ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.145002ms iteration = 20 status = "NoSitrep"
78972025-11-13T08:03:47.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
78982025-11-13T08:03:47.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
78992025-11-13T08:03:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
79002025-11-13T08:03:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
79012025-11-13T08:03:47.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79022025-11-13T08:03:47.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
79032025-11-13T08:03:47.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
79042025-11-13T08:03:47.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.307073ms iteration = 20 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
79052025-11-13T08:03:52.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 30 reason = Timeout
79062025-11-13T08:03:52.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 59 reason = Timeout
79072025-11-13T08:03:52.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79082025-11-13T08:03:52.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79092025-11-13T08:03:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79102025-11-13T08:03:52.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79112025-11-13T08:03:52.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79122025-11-13T08:03:52.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79132025-11-13T08:03:52.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
79142025-11-13T08:03:52.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79152025-11-13T08:03:52.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79162025-11-13T08:03:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79172025-11-13T08:03:52.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79182025-11-13T08:03:52.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79192025-11-13T08:03:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79202025-11-13T08:03:52.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79212025-11-13T08:03:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79222025-11-13T08:03:52.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
79232025-11-13T08:03:52.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.170471ms iteration = 59 status = {"config_updated":false}
79242025-11-13T08:03:52.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
79252025-11-13T08:03:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79262025-11-13T08:03:52.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
79272025-11-13T08:03:52.640ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
79282025-11-13T08:03:52.640ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.614414ms iteration = 30 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
79292025-11-13T08:03:52.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 30 reason = Timeout
79302025-11-13T08:03:52.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79312025-11-13T08:03:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79322025-11-13T08:03:52.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
79332025-11-13T08:03:52.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
79342025-11-13T08:03:52.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.948988ms iteration = 30 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
79352025-11-13T08:03:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 380 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 2b83cb46-0712-4547-b5d8-af084613a6f7 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
79362025-11-13T08:03:57.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 60 reason = Timeout
79372025-11-13T08:03:57.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79382025-11-13T08:03:57.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79392025-11-13T08:03:57.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79402025-11-13T08:03:57.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
79412025-11-13T08:03:57.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
79422025-11-13T08:03:57.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
79432025-11-13T08:03:57.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
79442025-11-13T08:03:57.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.181181ms iteration = 60 status = {"config_updated":false}
79452025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 6 reason = Timeout
79462025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 6 reason = Timeout
79472025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79482025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 6 reason = Timeout
79492025-11-13T08:04:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:54:02.607619696 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
79502025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
79512025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 11 reason = Timeout
79522025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 6 reason = Timeout
79532025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
79542025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 8 reason = Timeout
79552025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79562025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 11 reason = Timeout
79572025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
79582025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 6 reason = Timeout
79592025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
79602025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79612025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 8 reason = Timeout
79622025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 11 reason = Timeout
79632025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
79642025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 6 reason = Timeout
79652025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 7 reason = Timeout
79662025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
79672025-11-13T08:04:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
79682025-11-13T08:04:02.607ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = ExternalNetworking step id = Ensure
79692025-11-13T08:04:02.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
79702025-11-13T08:04:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
79712025-11-13T08:04:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
79722025-11-13T08:04:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79732025-11-13T08:04:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79742025-11-13T08:04:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
79752025-11-13T08:04:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
79762025-11-13T08:04:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
79772025-11-13T08:04:02.609ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6250, tv_nsec: 961855968 } })
79782025-11-13T08:04:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
79792025-11-13T08:04:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
79802025-11-13T08:04:02.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6250, tv_nsec: 962467870 } })
79812025-11-13T08:04:02.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
79822025-11-13T08:04:02.610ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
79832025-11-13T08:04:02.610ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.933417ms iteration = 6 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
79842025-11-13T08:04:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
79852025-11-13T08:04:02.610ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
79862025-11-13T08:04:02.610ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.874446ms iteration = 6 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
79872025-11-13T08:04:02.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79882025-11-13T08:04:02.616ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79892025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
79902025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
79912025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
79922025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 9 reason = Timeout
79932025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 18 reason = Timeout
79942025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 31 reason = Timeout
79952025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
79962025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
79972025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
79982025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 27 reason = Timeout
79992025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 61 reason = Timeout
80002025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
80012025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
80022025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 15 reason = Timeout
80032025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 6 reason = Timeout
80042025-11-13T08:04:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
80052025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
80062025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80072025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 16 reason = Timeout
80082025-11-13T08:04:02.618ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80092025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
80102025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
80112025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
80122025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
80132025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
80142025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
80152025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80162025-11-13T08:04:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80172025-11-13T08:04:02.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80182025-11-13T08:04:02.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80192025-11-13T08:04:02.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
80202025-11-13T08:04:02.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80212025-11-13T08:04:02.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80222025-11-13T08:04:02.624ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6250, tv_nsec: 976990416 } })
80232025-11-13T08:04:02.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80242025-11-13T08:04:02.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80252025-11-13T08:04:02.626ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80262025-11-13T08:04:02.626ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6250, tv_nsec: 979122618 } })
80272025-11-13T08:04:02.627ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
80282025-11-13T08:04:02.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80292025-11-13T08:04:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
80302025-11-13T08:04:02.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80312025-11-13T08:04:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
80322025-11-13T08:04:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
80332025-11-13T08:04:02.631ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
80342025-11-13T08:04:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
80352025-11-13T08:04:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
80362025-11-13T08:04:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
80372025-11-13T08:04:02.632ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
80382025-11-13T08:04:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
80392025-11-13T08:04:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
80402025-11-13T08:04:02.633ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
80412025-11-13T08:04:02.633ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
80422025-11-13T08:04:02.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
80432025-11-13T08:04:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
80442025-11-13T08:04:02.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
80452025-11-13T08:04:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
80462025-11-13T08:04:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
80472025-11-13T08:04:02.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
80482025-11-13T08:04:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
80492025-11-13T08:04:02.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80502025-11-13T08:04:02.638ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:54:02.607619696 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
80512025-11-13T08:04:02.638ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 30.579031ms iteration = 6 status = {"expiration":"2025-11-13T07:54:02.607619696Z","pruned":[]}
80522025-11-13T08:04:02.640ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
80532025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 2 reason = Timeout
80542025-11-13T08:04:02.642ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
80552025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
80562025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
80572025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 6 reason = Timeout
80582025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
80592025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
80602025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 11 reason = Timeout
80612025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
80622025-11-13T08:04:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
80632025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 11 reason = Timeout
80642025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 11 reason = Timeout
80652025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
80662025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
80672025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 11 reason = Timeout
80682025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
80692025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 11 reason = Timeout
80702025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 16 reason = Timeout
80712025-11-13T08:04:02.643ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
80722025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
80732025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
80742025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 11 reason = Timeout
80752025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 31 reason = Timeout
80762025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
80772025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
80782025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 6 reason = Timeout
80792025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
80802025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
80812025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
80822025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
80832025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
80842025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
80852025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
80862025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
80872025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
80882025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
80892025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
80902025-11-13T08:04:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
80912025-11-13T08:04:02.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80922025-11-13T08:04:02.644ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80932025-11-13T08:04:02.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80942025-11-13T08:04:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 6 reason = Timeout
80952025-11-13T08:04:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
80962025-11-13T08:04:02.645ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 38.15403ms iteration = 11 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
80972025-11-13T08:04:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
80982025-11-13T08:04:02.646ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
80992025-11-13T08:04:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
81002025-11-13T08:04:02.646ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81012025-11-13T08:04:02.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81022025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
81032025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 29.254516ms iteration = 61 status = {"config_updated":false}
81042025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 2 reason = Timeout
81052025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 11 reason = Timeout
81062025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
81072025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
81082025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 6 reason = Timeout
81092025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
81102025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
81112025-11-13T08:04:02.648ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
81122025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 2 reason = Timeout
81132025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 5.672032ms iteration = 6 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
81142025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
81152025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 11 reason = Timeout
81162025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
81172025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 6 reason = Timeout
81182025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 11 reason = Timeout
81192025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
81202025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
81212025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 11 reason = Timeout
81222025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 11 reason = Timeout
81232025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
81242025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 21 reason = Timeout
81252025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
81262025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 21 reason = Timeout
81272025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
81282025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 11 reason = Timeout
81292025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
81302025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
81312025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 6 reason = Timeout
81322025-11-13T08:04:02.648ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
81332025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
81342025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
81352025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
81362025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
81372025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
81382025-11-13T08:04:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
81392025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
81402025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
81412025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
81422025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
81432025-11-13T08:04:02.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6251, tv_nsec: 1632670 } })
81442025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
81452025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
81462025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.389057ms iteration = 6 status = {"errors":[],"found_port_ok":[]}
81472025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81482025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
81492025-11-13T08:04:02.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
81502025-11-13T08:04:02.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81512025-11-13T08:04:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
81522025-11-13T08:04:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
81532025-11-13T08:04:02.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
81542025-11-13T08:04:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
81552025-11-13T08:04:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
81562025-11-13T08:04:02.651ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
81572025-11-13T08:04:02.652ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
81582025-11-13T08:04:02.652ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81592025-11-13T08:04:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
81602025-11-13T08:04:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
81612025-11-13T08:04:02.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
81622025-11-13T08:04:02.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
81632025-11-13T08:04:02.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
81642025-11-13T08:04:02.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
81652025-11-13T08:04:02.653ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 5.139231ms iteration = 11 status = {"errors":[],"garbage_collect_requested":[]}
81662025-11-13T08:04:02.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
81672025-11-13T08:04:02.654ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
81682025-11-13T08:04:02.654ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 5.886396ms iteration = 6 status = {"by_rx":{},"error":null}
81692025-11-13T08:04:02.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
81702025-11-13T08:04:02.654ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
81712025-11-13T08:04:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
81722025-11-13T08:04:02.655ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 7.467537ms iteration = 11 status = {"errors":[],"requests_created_ok":[]}
81732025-11-13T08:04:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
81742025-11-13T08:04:02.655ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
81752025-11-13T08:04:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
81762025-11-13T08:04:02.655ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
81772025-11-13T08:04:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
81782025-11-13T08:04:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
81792025-11-13T08:04:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
81802025-11-13T08:04:02.657ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
81812025-11-13T08:04:02.657ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
81822025-11-13T08:04:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
81832025-11-13T08:04:02.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81842025-11-13T08:04:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
81852025-11-13T08:04:02.659ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 11.297552ms iteration = 11 status = {"errors":[],"finish_invoked_ok":[]}
81862025-11-13T08:04:02.659ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
81872025-11-13T08:04:02.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
81882025-11-13T08:04:02.660ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
81892025-11-13T08:04:02.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
81902025-11-13T08:04:02.661ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
81912025-11-13T08:04:02.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
81922025-11-13T08:04:02.664ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
81932025-11-13T08:04:02.665ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
81942025-11-13T08:04:02.665ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 22.469221ms iteration = 11 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
81952025-11-13T08:04:02.666ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
81962025-11-13T08:04:02.666ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81972025-11-13T08:04:02.666ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81982025-11-13T08:04:02.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
81992025-11-13T08:04:02.667ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
82002025-11-13T08:04:02.669ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7 original_time_started = 2025-11-13 08:03:04.254043 UTC
82012025-11-13T08:04:02.669ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 50.560054ms iteration = 27 status = {"Loaded":{"collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","time_loaded":"2025-11-13T08:04:02.618762056Z","time_started":"2025-11-13T08:03:04.254043Z"}}
82022025-11-13T08:04:02.669ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82032025-11-13T08:04:02.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82042025-11-13T08:04:02.670ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82052025-11-13T08:04:02.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
82062025-11-13T08:04:02.670ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
82072025-11-13T08:04:02.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
82082025-11-13T08:04:02.671ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
82092025-11-13T08:04:02.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
82102025-11-13T08:04:02.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
82112025-11-13T08:04:02.675ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
82122025-11-13T08:04:02.675ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
82132025-11-13T08:04:02.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
82142025-11-13T08:04:02.676ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
82152025-11-13T08:04:02.677ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82162025-11-13T08:04:02.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
82172025-11-13T08:04:02.682ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
82182025-11-13T08:04:02.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
82192025-11-13T08:04:02.683ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82202025-11-13T08:04:02.686ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
82212025-11-13T08:04:02.688ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
82222025-11-13T08:04:02.691ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
82232025-11-13T08:04:02.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
82242025-11-13T08:04:02.691ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82252025-11-13T08:04:02.695ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6251, tv_nsec: 48074153 } })
82262025-11-13T08:04:02.696ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
82272025-11-13T08:04:02.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
82282025-11-13T08:04:02.697ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
82292025-11-13T08:04:02.697ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
82302025-11-13T08:04:02.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
82312025-11-13T08:04:02.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
82322025-11-13T08:04:02.702ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 54.127244ms iteration = 21 status = "NoSitrep"
82332025-11-13T08:04:02.702ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82342025-11-13T08:04:02.703ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82352025-11-13T08:04:02.703ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
82362025-11-13T08:04:02.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82372025-11-13T08:04:02.706ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82382025-11-13T08:04:02.707ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
82392025-11-13T08:04:02.707ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
82402025-11-13T08:04:02.707ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
82412025-11-13T08:04:02.708ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82422025-11-13T08:04:02.710ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
82432025-11-13T08:04:02.710ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
82442025-11-13T08:04:02.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
82452025-11-13T08:04:02.712ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
82462025-11-13T08:04:02.712ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
82472025-11-13T08:04:02.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
82482025-11-13T08:04:02.713ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82492025-11-13T08:04:02.717ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
82502025-11-13T08:04:02.717ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
82512025-11-13T08:04:02.717ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
82522025-11-13T08:04:02.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
82532025-11-13T08:04:02.717ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 74.650778ms iteration = 11 status = 0
82542025-11-13T08:04:02.718ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
82552025-11-13T08:04:02.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
82562025-11-13T08:04:02.724ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 76.142377ms iteration = 6 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
82572025-11-13T08:04:02.728ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
82582025-11-13T08:04:02.730ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82592025-11-13T08:04:02.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82602025-11-13T08:04:02.731ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
82612025-11-13T08:04:02.735ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
82622025-11-13T08:04:02.735ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
82632025-11-13T08:04:02.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
82642025-11-13T08:04:02.738ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
82652025-11-13T08:04:02.740ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
82662025-11-13T08:04:02.740ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 121.629471ms iteration = 31 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
82672025-11-13T08:04:02.741ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
82682025-11-13T08:04:02.741ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = SledList step id = Fetch
82692025-11-13T08:04:02.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82702025-11-13T08:04:02.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82712025-11-13T08:04:02.743ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
82722025-11-13T08:04:02.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
82732025-11-13T08:04:02.744ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82742025-11-13T08:04:02.745ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82752025-11-13T08:04:02.746ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
82762025-11-13T08:04:02.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
82772025-11-13T08:04:02.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82782025-11-13T08:04:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
82792025-11-13T08:04:02.749ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82802025-11-13T08:04:02.750ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
82812025-11-13T08:04:02.750ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 101.550416ms iteration = 21 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
82822025-11-13T08:04:02.753ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
82832025-11-13T08:04:02.753ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
82842025-11-13T08:04:02.753ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
82852025-11-13T08:04:02.753ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
82862025-11-13T08:04:02.753ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
82872025-11-13T08:04:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
82882025-11-13T08:04:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
82892025-11-13T08:04:02.754ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
82902025-11-13T08:04:02.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
82912025-11-13T08:04:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
82922025-11-13T08:04:02.755ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
82932025-11-13T08:04:02.755ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
82942025-11-13T08:04:02.755ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
82952025-11-13T08:04:02.755ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
82962025-11-13T08:04:02.755ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
82972025-11-13T08:04:02.755ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
82982025-11-13T08:04:02.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
82992025-11-13T08:04:02.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
83002025-11-13T08:04:02.756ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
83012025-11-13T08:04:02.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
83022025-11-13T08:04:02.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
83032025-11-13T08:04:02.757ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83042025-11-13T08:04:02.758ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:114 status = TufRepoPrunerStatus { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
83052025-11-13T08:04:02.758ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 110.726057ms iteration = 2 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"warnings":[]}
83062025-11-13T08:04:02.762ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83072025-11-13T08:04:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83082025-11-13T08:04:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
83092025-11-13T08:04:02.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
83102025-11-13T08:04:02.763ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83112025-11-13T08:04:02.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
83122025-11-13T08:04:02.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83132025-11-13T08:04:02.766ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83142025-11-13T08:04:02.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83152025-11-13T08:04:02.768ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83162025-11-13T08:04:02.769ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
83172025-11-13T08:04:02.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83182025-11-13T08:04:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83192025-11-13T08:04:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
83202025-11-13T08:04:02.770ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
83212025-11-13T08:04:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
83222025-11-13T08:04:02.773ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
83232025-11-13T08:04:02.773ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
83242025-11-13T08:04:02.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
83252025-11-13T08:04:02.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83262025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
83272025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83282025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
83292025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
83302025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
83312025-11-13T08:04:02.778ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = DeployNexusRecords step id = Ensure
83322025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83332025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
83342025-11-13T08:04:02.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
83352025-11-13T08:04:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83362025-11-13T08:04:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
83372025-11-13T08:04:02.779ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
83382025-11-13T08:04:02.779ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
83392025-11-13T08:04:02.779ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
83402025-11-13T08:04:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
83412025-11-13T08:04:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
83422025-11-13T08:04:02.780ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
83432025-11-13T08:04:02.780ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
83442025-11-13T08:04:02.780ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
83452025-11-13T08:04:02.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
83462025-11-13T08:04:02.781ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 132.940443ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
83472025-11-13T08:04:02.781ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83482025-11-13T08:04:02.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
83492025-11-13T08:04:02.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
83502025-11-13T08:04:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
83512025-11-13T08:04:02.782ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
83522025-11-13T08:04:02.782ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83532025-11-13T08:04:02.782ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
83542025-11-13T08:04:02.783ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
83552025-11-13T08:04:02.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
83562025-11-13T08:04:02.783ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83572025-11-13T08:04:02.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83582025-11-13T08:04:02.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83592025-11-13T08:04:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
83602025-11-13T08:04:02.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83612025-11-13T08:04:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83622025-11-13T08:04:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
83632025-11-13T08:04:02.787ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83642025-11-13T08:04:02.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83652025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
83662025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
83672025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
83682025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
83692025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
83702025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
83712025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
83722025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
83732025-11-13T08:04:02.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
83742025-11-13T08:04:02.790ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
83752025-11-13T08:04:02.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
83762025-11-13T08:04:02.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
83772025-11-13T08:04:02.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
83782025-11-13T08:04:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
83792025-11-13T08:04:02.791ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
83802025-11-13T08:04:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
83812025-11-13T08:04:02.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83822025-11-13T08:04:02.793ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
83832025-11-13T08:04:02.793ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
83842025-11-13T08:04:02.793ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
83852025-11-13T08:04:02.793ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 186.269552ms iteration = 6 status = {"generation":1}
83862025-11-13T08:04:02.794ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
83872025-11-13T08:04:02.794ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
83882025-11-13T08:04:02.794ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
83892025-11-13T08:04:02.794ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 187.14744ms iteration = 6 status = {"generation":2}
83902025-11-13T08:04:02.795ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
83912025-11-13T08:04:02.795ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
83922025-11-13T08:04:02.795ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 176.666323ms iteration = 6 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
83932025-11-13T08:04:02.795ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83942025-11-13T08:04:02.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
83952025-11-13T08:04:02.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
83962025-11-13T08:04:02.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
83972025-11-13T08:04:02.801ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
83982025-11-13T08:04:02.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
83992025-11-13T08:04:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84002025-11-13T08:04:02.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84012025-11-13T08:04:02.809ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
84022025-11-13T08:04:02.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84032025-11-13T08:04:02.809ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84042025-11-13T08:04:02.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84052025-11-13T08:04:02.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84062025-11-13T08:04:02.812ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84072025-11-13T08:04:02.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84082025-11-13T08:04:02.814ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84092025-11-13T08:04:02.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84102025-11-13T08:04:02.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84112025-11-13T08:04:02.819ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
84122025-11-13T08:04:02.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84132025-11-13T08:04:02.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84142025-11-13T08:04:02.820ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84152025-11-13T08:04:02.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84162025-11-13T08:04:02.823ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84172025-11-13T08:04:02.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
84182025-11-13T08:04:02.825ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84192025-11-13T08:04:02.826ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
84202025-11-13T08:04:02.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84212025-11-13T08:04:02.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84222025-11-13T08:04:02.829ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84232025-11-13T08:04:02.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84242025-11-13T08:04:02.832ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 189.653429ms iteration = 31 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
84252025-11-13T08:04:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
84262025-11-13T08:04:02.833ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84272025-11-13T08:04:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84282025-11-13T08:04:02.834ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
84292025-11-13T08:04:02.834ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84302025-11-13T08:04:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
84312025-11-13T08:04:02.835ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
84322025-11-13T08:04:02.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84332025-11-13T08:04:02.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84342025-11-13T08:04:02.835ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84352025-11-13T08:04:02.836ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84362025-11-13T08:04:02.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84372025-11-13T08:04:02.836ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84382025-11-13T08:04:02.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84392025-11-13T08:04:02.839ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84402025-11-13T08:04:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84412025-11-13T08:04:02.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
84422025-11-13T08:04:02.840ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
84432025-11-13T08:04:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
84442025-11-13T08:04:02.841ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 198.636045ms iteration = 11 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
84452025-11-13T08:04:02.842ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
84462025-11-13T08:04:02.842ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
84472025-11-13T08:04:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
84482025-11-13T08:04:02.842ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
84492025-11-13T08:04:02.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
84502025-11-13T08:04:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
84512025-11-13T08:04:02.844ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84522025-11-13T08:04:02.845ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84532025-11-13T08:04:02.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84542025-11-13T08:04:02.847ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
84552025-11-13T08:04:02.847ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 228.786328ms iteration = 9 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
84562025-11-13T08:04:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
84572025-11-13T08:04:02.848ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
84582025-11-13T08:04:02.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
84592025-11-13T08:04:02.849ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84602025-11-13T08:04:02.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84612025-11-13T08:04:02.851ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84622025-11-13T08:04:02.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84632025-11-13T08:04:02.854ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
84642025-11-13T08:04:02.854ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 246.7378ms iteration = 6 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
84652025-11-13T08:04:02.858ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84662025-11-13T08:04:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84672025-11-13T08:04:02.861ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84682025-11-13T08:04:02.868ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
84692025-11-13T08:04:02.869ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84702025-11-13T08:04:02.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84712025-11-13T08:04:02.871ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84722025-11-13T08:04:02.872ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84732025-11-13T08:04:02.873ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
84742025-11-13T08:04:02.873ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
84752025-11-13T08:04:02.873ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 230.870199ms iteration = 11 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
84762025-11-13T08:04:02.874ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84772025-11-13T08:04:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84782025-11-13T08:04:02.876ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84792025-11-13T08:04:02.879ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
84802025-11-13T08:04:02.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
84812025-11-13T08:04:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
84822025-11-13T08:04:02.884ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
84832025-11-13T08:04:02.884ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84842025-11-13T08:04:02.884ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 236.246375ms iteration = 11 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
84852025-11-13T08:04:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84862025-11-13T08:04:02.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
84872025-11-13T08:04:02.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
84882025-11-13T08:04:02.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84892025-11-13T08:04:02.897ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
84902025-11-13T08:04:02.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
84912025-11-13T08:04:02.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
84922025-11-13T08:04:02.898ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84932025-11-13T08:04:02.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84942025-11-13T08:04:02.899ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84952025-11-13T08:04:02.901ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = SledAgent step id = Ensure
84962025-11-13T08:04:02.901ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
84972025-11-13T08:04:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
84982025-11-13T08:04:02.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
84992025-11-13T08:04:02.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85002025-11-13T08:04:02.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85012025-11-13T08:04:02.905ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85022025-11-13T08:04:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85032025-11-13T08:04:02.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85042025-11-13T08:04:02.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85052025-11-13T08:04:02.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85062025-11-13T08:04:02.914ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85072025-11-13T08:04:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85082025-11-13T08:04:02.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85092025-11-13T08:04:02.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85102025-11-13T08:04:02.917ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85112025-11-13T08:04:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85122025-11-13T08:04:02.919ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85132025-11-13T08:04:02.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85142025-11-13T08:04:02.921ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85152025-11-13T08:04:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85162025-11-13T08:04:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
85172025-11-13T08:04:02.922ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85182025-11-13T08:04:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85192025-11-13T08:04:02.924ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
85202025-11-13T08:04:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
85212025-11-13T08:04:02.924ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85222025-11-13T08:04:02.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85232025-11-13T08:04:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85242025-11-13T08:04:02.932ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
85252025-11-13T08:04:02.932ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 313.443922ms iteration = 16 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
85262025-11-13T08:04:02.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
85272025-11-13T08:04:02.932ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85282025-11-13T08:04:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85292025-11-13T08:04:02.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85302025-11-13T08:04:02.936ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85312025-11-13T08:04:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85322025-11-13T08:04:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
85332025-11-13T08:04:02.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85342025-11-13T08:04:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85352025-11-13T08:04:02.939ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85362025-11-13T08:04:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85372025-11-13T08:04:02.940ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
85382025-11-13T08:04:02.941ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
85392025-11-13T08:04:02.941ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
85402025-11-13T08:04:02.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85412025-11-13T08:04:02.941ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f07e2ffd-ada5-4140-8748-84b6e1e866c7", "content-length": "26", "date": "Thu, 13 Nov 2025 08:04:02 GMT"} })
85422025-11-13T08:04:02.941ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
85432025-11-13T08:04:02.942ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85442025-11-13T08:04:02.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
85452025-11-13T08:04:02.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85462025-11-13T08:04:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85472025-11-13T08:04:02.950ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
85482025-11-13T08:04:02.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85492025-11-13T08:04:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85502025-11-13T08:04:02.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
85512025-11-13T08:04:02.951ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85522025-11-13T08:04:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85532025-11-13T08:04:02.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85542025-11-13T08:04:02.957ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
85552025-11-13T08:04:02.957ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
85562025-11-13T08:04:02.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85572025-11-13T08:04:02.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85582025-11-13T08:04:02.958ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d272722-5dba-411b-b885-d38b014beb75", "content-length": "78", "date": "Thu, 13 Nov 2025 08:04:02 GMT"} })
85592025-11-13T08:04:02.958ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
85602025-11-13T08:04:02.958ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
85612025-11-13T08:04:02.958ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
85622025-11-13T08:04:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
85632025-11-13T08:04:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
85642025-11-13T08:04:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
85652025-11-13T08:04:02.960ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
85662025-11-13T08:04:02.960ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
85672025-11-13T08:04:02.960ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
85682025-11-13T08:04:02.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85692025-11-13T08:04:02.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85702025-11-13T08:04:02.962ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
85712025-11-13T08:04:02.962ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85722025-11-13T08:04:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85732025-11-13T08:04:02.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85742025-11-13T08:04:02.964ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85752025-11-13T08:04:02.965ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
85762025-11-13T08:04:02.965ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
85772025-11-13T08:04:02.965ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
85782025-11-13T08:04:02.965ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
85792025-11-13T08:04:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
85802025-11-13T08:04:02.966ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
85812025-11-13T08:04:02.966ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
85822025-11-13T08:04:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
85832025-11-13T08:04:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
85842025-11-13T08:04:02.968ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
85852025-11-13T08:04:02.968ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
85862025-11-13T08:04:02.968ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
85872025-11-13T08:04:02.968ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
85882025-11-13T08:04:02.969ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
85892025-11-13T08:04:02.969ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
85902025-11-13T08:04:02.969ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "377fda27-b672-4d5b-8e8a-62b53e2f2113", "content-length": "81", "date": "Thu, 13 Nov 2025 08:04:02 GMT"} })
85912025-11-13T08:04:02.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
85922025-11-13T08:04:02.969ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "aabc8b5a-ad4f-42c0-9583-e19a0dea52b7", "content-length": "81", "date": "Thu, 13 Nov 2025 08:04:02 GMT"} })
85932025-11-13T08:04:02.970ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
85942025-11-13T08:04:02.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85952025-11-13T08:04:02.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85962025-11-13T08:04:02.971ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
85972025-11-13T08:04:02.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
85982025-11-13T08:04:02.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
85992025-11-13T08:04:02.974ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
86002025-11-13T08:04:02.974ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
86012025-11-13T08:04:02.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86022025-11-13T08:04:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
86032025-11-13T08:04:02.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86042025-11-13T08:04:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86052025-11-13T08:04:03.016ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
86062025-11-13T08:04:03.016ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 373.435832ms iteration = 6 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
86072025-11-13T08:04:03.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86082025-11-13T08:04:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86092025-11-13T08:04:03.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86102025-11-13T08:04:03.028ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86112025-11-13T08:04:03.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86122025-11-13T08:04:03.036ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86132025-11-13T08:04:03.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86142025-11-13T08:04:03.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86152025-11-13T08:04:03.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86162025-11-13T08:04:03.041ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
86172025-11-13T08:04:03.041ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
86182025-11-13T08:04:03.042ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "a84d7118-4529-4f62-9046-94bee97bb8f2", "content-length": "81", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
86192025-11-13T08:04:03.042ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 423.921484ms iteration = 18 status = {"physical_disks_added":0}
86202025-11-13T08:04:03.044ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 395.949254ms iteration = 11 status = {"disabled":false,"errors":[],"sps":[]}
86212025-11-13T08:04:03.045ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86222025-11-13T08:04:03.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86232025-11-13T08:04:03.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86242025-11-13T08:04:03.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86252025-11-13T08:04:03.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86262025-11-13T08:04:03.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86272025-11-13T08:04:03.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86282025-11-13T08:04:03.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86292025-11-13T08:04:03.060ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86302025-11-13T08:04:03.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86312025-11-13T08:04:03.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86322025-11-13T08:04:03.063ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86332025-11-13T08:04:03.066ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
86342025-11-13T08:04:03.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86352025-11-13T08:04:03.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86362025-11-13T08:04:03.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86372025-11-13T08:04:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86382025-11-13T08:04:03.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86392025-11-13T08:04:03.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86402025-11-13T08:04:03.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86412025-11-13T08:04:03.083ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
86422025-11-13T08:04:03.089ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
86432025-11-13T08:04:03.089ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
86442025-11-13T08:04:03.117ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
86452025-11-13T08:04:03.117ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
86462025-11-13T08:04:03.117ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "86996881-a682-4a59-8563-2bda9a640cce", "content-length": "26", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
86472025-11-13T08:04:03.117ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
86482025-11-13T08:04:03.120ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
86492025-11-13T08:04:03.122ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
86502025-11-13T08:04:03.124ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 516.399762ms iteration = 11 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
86512025-11-13T08:04:03.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86522025-11-13T08:04:03.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86532025-11-13T08:04:03.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86542025-11-13T08:04:03.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86552025-11-13T08:04:03.152ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86562025-11-13T08:04:03.153ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86572025-11-13T08:04:03.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86582025-11-13T08:04:03.155ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86592025-11-13T08:04:03.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
86602025-11-13T08:04:03.159ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86612025-11-13T08:04:03.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86622025-11-13T08:04:03.159ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
86632025-11-13T08:04:03.160ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86642025-11-13T08:04:03.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86652025-11-13T08:04:03.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86662025-11-13T08:04:03.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86672025-11-13T08:04:03.163ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86682025-11-13T08:04:03.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86692025-11-13T08:04:03.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86702025-11-13T08:04:03.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86712025-11-13T08:04:03.167ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
86722025-11-13T08:04:03.167ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86732025-11-13T08:04:03.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86742025-11-13T08:04:03.170ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86752025-11-13T08:04:03.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86762025-11-13T08:04:03.170ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
86772025-11-13T08:04:03.170ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86782025-11-13T08:04:03.170ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86792025-11-13T08:04:03.171ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86802025-11-13T08:04:03.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86812025-11-13T08:04:03.172ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86822025-11-13T08:04:03.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86832025-11-13T08:04:03.175ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
86842025-11-13T08:04:03.175ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
86852025-11-13T08:04:03.175ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
86862025-11-13T08:04:03.179ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
86872025-11-13T08:04:03.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86882025-11-13T08:04:03.180ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86892025-11-13T08:04:03.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
86902025-11-13T08:04:03.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
86912025-11-13T08:04:03.187ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
86922025-11-13T08:04:03.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86932025-11-13T08:04:03.189ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "31307114-4628-4fd3-9554-f599e6bcdbd8", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
86942025-11-13T08:04:03.189ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
86952025-11-13T08:04:03.190ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "2a53ec90-0633-44f4-b1c0-6f2f2f422777", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
86962025-11-13T08:04:03.190ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
86972025-11-13T08:04:03.190ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "24c41335-a26e-4e61-a2bf-8d165ba3bbe9", "content-length": "26", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
86982025-11-13T08:04:03.190ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:679 sled = http://[fd00:1122:3344:101::1]:12345
86992025-11-13T08:04:03.191ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = FirewallRules step id = Ensure
87002025-11-13T08:04:03.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87012025-11-13T08:04:03.191ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
87022025-11-13T08:04:03.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87032025-11-13T08:04:03.191ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
87042025-11-13T08:04:03.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87052025-11-13T08:04:03.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87062025-11-13T08:04:03.192ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "1699b5b3-7112-4ed4-9759-9e87bfa30532", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
87072025-11-13T08:04:03.192ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
87082025-11-13T08:04:03.193ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 545.612266ms iteration = 2 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"370ec2da-8f7f-4861-9b77-369c2bb1c86d","time":"2025-11-13T08:04:03.190504056Z"},{"operation":"list","target_sled":"370ec2da-8f7f-4861-9b77-369c2bb1c86d","time":"2025-11-13T08:04:03.190815752Z"}]}
87092025-11-13T08:04:03.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87102025-11-13T08:04:03.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87112025-11-13T08:04:03.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87122025-11-13T08:04:03.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87132025-11-13T08:04:03.199ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87142025-11-13T08:04:03.200ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
87152025-11-13T08:04:03.200ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 551.37894ms iteration = 6 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
87162025-11-13T08:04:03.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87172025-11-13T08:04:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
87182025-11-13T08:04:03.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87192025-11-13T08:04:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87202025-11-13T08:04:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87212025-11-13T08:04:03.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87222025-11-13T08:04:03.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87232025-11-13T08:04:03.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87242025-11-13T08:04:03.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87252025-11-13T08:04:03.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87262025-11-13T08:04:03.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87272025-11-13T08:04:03.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87282025-11-13T08:04:03.218ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
87292025-11-13T08:04:03.218ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87302025-11-13T08:04:03.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87312025-11-13T08:04:03.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87322025-11-13T08:04:03.220ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
87332025-11-13T08:04:03.221ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "ea98c422-9fee-4b7f-bc9f-86a833e60393", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87342025-11-13T08:04:03.221ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87352025-11-13T08:04:03.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87362025-11-13T08:04:03.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87372025-11-13T08:04:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87382025-11-13T08:04:03.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87392025-11-13T08:04:03.227ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "229fe002-fa64-4a26-bcb9-f91b02358aa9", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87402025-11-13T08:04:03.227ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87412025-11-13T08:04:03.229ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
87422025-11-13T08:04:03.229ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 621.902316ms iteration = 8 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
87432025-11-13T08:04:03.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
87442025-11-13T08:04:03.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87452025-11-13T08:04:03.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87462025-11-13T08:04:03.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87472025-11-13T08:04:03.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87482025-11-13T08:04:03.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87492025-11-13T08:04:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87502025-11-13T08:04:03.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87512025-11-13T08:04:03.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87522025-11-13T08:04:03.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87532025-11-13T08:04:03.242ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87542025-11-13T08:04:03.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87552025-11-13T08:04:03.244ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87562025-11-13T08:04:03.245ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87572025-11-13T08:04:03.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87582025-11-13T08:04:03.247ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87592025-11-13T08:04:03.251ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
87602025-11-13T08:04:03.251ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
87612025-11-13T08:04:03.252ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 98f70f13-1d11-4206-af92-34562ad2ddfe parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
87622025-11-13T08:04:03.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87632025-11-13T08:04:03.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87642025-11-13T08:04:03.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
87652025-11-13T08:04:03.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87662025-11-13T08:04:03.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87672025-11-13T08:04:03.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87682025-11-13T08:04:03.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87692025-11-13T08:04:03.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87702025-11-13T08:04:03.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87712025-11-13T08:04:03.259ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
87722025-11-13T08:04:03.259ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
87732025-11-13T08:04:03.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87742025-11-13T08:04:03.259ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
87752025-11-13T08:04:03.261ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87762025-11-13T08:04:03.261ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87772025-11-13T08:04:03.262ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
87782025-11-13T08:04:03.262ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
87792025-11-13T08:04:03.262ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 643.587213ms iteration = 15 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
87802025-11-13T08:04:03.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87812025-11-13T08:04:03.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87822025-11-13T08:04:03.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87832025-11-13T08:04:03.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87842025-11-13T08:04:03.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87852025-11-13T08:04:03.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87862025-11-13T08:04:03.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87872025-11-13T08:04:03.275ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
87882025-11-13T08:04:03.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87892025-11-13T08:04:03.275ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87902025-11-13T08:04:03.276ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87912025-11-13T08:04:03.276ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87922025-11-13T08:04:03.278ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87932025-11-13T08:04:03.278ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87942025-11-13T08:04:03.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87952025-11-13T08:04:03.286ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
87962025-11-13T08:04:03.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87972025-11-13T08:04:03.287ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
87982025-11-13T08:04:03.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
87992025-11-13T08:04:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88002025-11-13T08:04:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88012025-11-13T08:04:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88022025-11-13T08:04:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88032025-11-13T08:04:03.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88042025-11-13T08:04:03.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88052025-11-13T08:04:03.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88062025-11-13T08:04:03.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88072025-11-13T08:04:03.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88082025-11-13T08:04:03.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88092025-11-13T08:04:03.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88102025-11-13T08:04:03.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88112025-11-13T08:04:03.307ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88122025-11-13T08:04:03.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88132025-11-13T08:04:03.314ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
88142025-11-13T08:04:03.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88152025-11-13T08:04:03.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88162025-11-13T08:04:03.316ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88172025-11-13T08:04:03.316ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88182025-11-13T08:04:03.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88192025-11-13T08:04:03.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88202025-11-13T08:04:03.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88212025-11-13T08:04:03.327ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
88222025-11-13T08:04:03.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88232025-11-13T08:04:03.328ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88242025-11-13T08:04:03.334ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
88252025-11-13T08:04:03.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88262025-11-13T08:04:03.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88272025-11-13T08:04:03.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88282025-11-13T08:04:03.336ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
88292025-11-13T08:04:03.336ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
88302025-11-13T08:04:03.337ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "94e63cad-9e4e-4ddb-8635-68979d5d9240", "content-length": "372", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
88312025-11-13T08:04:03.337ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88322025-11-13T08:04:03.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88332025-11-13T08:04:03.338ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
88342025-11-13T08:04:03.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
88352025-11-13T08:04:03.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
88362025-11-13T08:04:03.340ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
88372025-11-13T08:04:03.340ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
88382025-11-13T08:04:03.340ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
88392025-11-13T08:04:03.359ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
88402025-11-13T08:04:03.359ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
88412025-11-13T08:04:03.360ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6662538d-83bb-4301-b783-9cb3bc59ee87", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
88422025-11-13T08:04:03.360ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88432025-11-13T08:04:03.360ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88442025-11-13T08:04:03.362ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88452025-11-13T08:04:03.363ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88462025-11-13T08:04:03.366ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88472025-11-13T08:04:03.366ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88482025-11-13T08:04:03.369ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
88492025-11-13T08:04:03.369ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
88502025-11-13T08:04:03.370ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "3f30f7dd-6520-4ad3-b83e-3df1ec77dfa5", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
88512025-11-13T08:04:03.372ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 729.379019ms iteration = 16 status = {}
88522025-11-13T08:04:03.372ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 17 reason = Dependency
88532025-11-13T08:04:03.372ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
88542025-11-13T08:04:03.372ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88552025-11-13T08:04:03.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88562025-11-13T08:04:03.374ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88572025-11-13T08:04:03.377ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
88582025-11-13T08:04:03.377ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88592025-11-13T08:04:03.377ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88602025-11-13T08:04:03.379ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88612025-11-13T08:04:03.381ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
88622025-11-13T08:04:03.381ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88632025-11-13T08:04:03.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88642025-11-13T08:04:03.389ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
88652025-11-13T08:04:03.390ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a86a512e-53fe-4584-af6b-69f6fcdeb155", "content-length": "691", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
88662025-11-13T08:04:03.390ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
88672025-11-13T08:04:03.390ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
88682025-11-13T08:04:03.390ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
88692025-11-13T08:04:03.392ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 749.452603ms iteration = 11 status = {}
88702025-11-13T08:04:03.404ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
88712025-11-13T08:04:03.404ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
88722025-11-13T08:04:03.406ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
88732025-11-13T08:04:03.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
88742025-11-13T08:04:03.409ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88752025-11-13T08:04:03.412ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
88762025-11-13T08:04:03.412ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
88772025-11-13T08:04:03.412ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
88782025-11-13T08:04:03.415ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
88792025-11-13T08:04:03.416ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "bd64d47c-48de-427c-a6b1-2a1c466c5f45", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
88802025-11-13T08:04:03.418ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
88812025-11-13T08:04:03.418ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 776.509905ms
88822025-11-13T08:04:03.418ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 776.524705ms iteration = 2 status = {}
88832025-11-13T08:04:03.450ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
88842025-11-13T08:04:03.515ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6251, tv_nsec: 868045223 } })
88852025-11-13T08:04:03.520ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
88862025-11-13T08:04:03.520ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
88872025-11-13T08:04:03.520ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "bec0b7f8-91ab-4992-8240-7aef575295ed", "content-length": "372", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
88882025-11-13T08:04:03.521ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88892025-11-13T08:04:03.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88902025-11-13T08:04:03.539ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
88912025-11-13T08:04:03.539ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
88922025-11-13T08:04:03.540ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3873eb7e-3065-4bf5-bb5f-7d9b2fe7c814", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
88932025-11-13T08:04:03.540ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88942025-11-13T08:04:03.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88952025-11-13T08:04:03.543ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88962025-11-13T08:04:03.543ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88972025-11-13T08:04:03.547ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
88982025-11-13T08:04:03.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
88992025-11-13T08:04:03.550ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
89002025-11-13T08:04:03.550ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
89012025-11-13T08:04:03.551ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "5e16298a-6c98-4259-a490-6e0f58d8fc2f", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
89022025-11-13T08:04:03.556ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 183.869225ms iteration = 17 status = {}
89032025-11-13T08:04:03.556ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 949.092769ms iteration = 11 status = {}
89042025-11-13T08:04:03.565ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89052025-11-13T08:04:03.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89062025-11-13T08:04:03.569ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89072025-11-13T08:04:03.569ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89082025-11-13T08:04:03.571ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89092025-11-13T08:04:03.575ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
89102025-11-13T08:04:03.575ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
89112025-11-13T08:04:03.576ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "4f178699-ba58-4635-9fda-135a02ca492e", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
89122025-11-13T08:04:03.578ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
89132025-11-13T08:04:03.578ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = Dns step id = Ensure
89142025-11-13T08:04:03.578ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
89152025-11-13T08:04:03.578ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89162025-11-13T08:04:03.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89172025-11-13T08:04:03.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89182025-11-13T08:04:03.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89192025-11-13T08:04:03.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89202025-11-13T08:04:03.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89212025-11-13T08:04:03.587ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
89222025-11-13T08:04:03.587ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
89232025-11-13T08:04:03.587ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
89242025-11-13T08:04:03.587ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89252025-11-13T08:04:03.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89262025-11-13T08:04:03.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89272025-11-13T08:04:03.594ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89282025-11-13T08:04:03.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89292025-11-13T08:04:03.594ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89302025-11-13T08:04:03.597ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
89312025-11-13T08:04:03.597ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
89322025-11-13T08:04:03.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89332025-11-13T08:04:03.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89342025-11-13T08:04:03.599ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
89352025-11-13T08:04:03.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89362025-11-13T08:04:03.600ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "22e2b4c8-baac-45da-9288-08e26985573b", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
89372025-11-13T08:04:03.600ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
89382025-11-13T08:04:03.603ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "7dc2c3a3-1358-4906-a42d-7fffa5711d97", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
89392025-11-13T08:04:03.603ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
89402025-11-13T08:04:03.605ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89412025-11-13T08:04:03.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89422025-11-13T08:04:03.605ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89432025-11-13T08:04:03.606ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "637f1053-3c93-45f9-9307-ae7e8c742ba8", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
89442025-11-13T08:04:03.606ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
89452025-11-13T08:04:03.608ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
89462025-11-13T08:04:03.608ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
89472025-11-13T08:04:03.608ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
89482025-11-13T08:04:03.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89492025-11-13T08:04:03.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89502025-11-13T08:04:03.609ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
89512025-11-13T08:04:03.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 1.001620311s iteration = 8 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
89522025-11-13T08:04:03.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89532025-11-13T08:04:03.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89542025-11-13T08:04:03.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89552025-11-13T08:04:03.616ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89562025-11-13T08:04:03.618ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
89572025-11-13T08:04:03.618ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
89582025-11-13T08:04:03.618ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
89592025-11-13T08:04:03.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89602025-11-13T08:04:03.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89612025-11-13T08:04:03.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89622025-11-13T08:04:03.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89632025-11-13T08:04:03.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89642025-11-13T08:04:03.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89652025-11-13T08:04:03.626ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
89662025-11-13T08:04:03.626ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
89672025-11-13T08:04:03.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89682025-11-13T08:04:03.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89692025-11-13T08:04:03.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89702025-11-13T08:04:03.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89712025-11-13T08:04:03.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89722025-11-13T08:04:03.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89732025-11-13T08:04:03.634ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
89742025-11-13T08:04:03.634ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
89752025-11-13T08:04:03.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89762025-11-13T08:04:03.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89772025-11-13T08:04:03.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89782025-11-13T08:04:03.638ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
89792025-11-13T08:04:03.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89802025-11-13T08:04:03.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89812025-11-13T08:04:03.639ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89822025-11-13T08:04:03.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89832025-11-13T08:04:03.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89842025-11-13T08:04:03.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89852025-11-13T08:04:03.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
89862025-11-13T08:04:03.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
89872025-11-13T08:04:03.642ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
89882025-11-13T08:04:03.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
89892025-11-13T08:04:03.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
89902025-11-13T08:04:03.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89912025-11-13T08:04:03.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
89922025-11-13T08:04:03.647ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
89932025-11-13T08:04:03.647ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
89942025-11-13T08:04:03.647ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = OmicronZones step id = Cleanup
89952025-11-13T08:04:03.647ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = OmicronZones step id = Cleanup
89962025-11-13T08:04:03.647ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = PhysicalDisks step id = Cleanup
89972025-11-13T08:04:03.647ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = Clickhouse step id = Ensure
89982025-11-13T08:04:03.647ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = Clickhouse step id = Ensure
89992025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
90002025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
90012025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
90022025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
90032025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90042025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90052025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90062025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
90072025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
90082025-11-13T08:04:03.652ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
90092025-11-13T08:04:03.654ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
90102025-11-13T08:04:03.654ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90112025-11-13T08:04:03.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90122025-11-13T08:04:03.656ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
90132025-11-13T08:04:03.656ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
90142025-11-13T08:04:03.677ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
90152025-11-13T08:04:03.682ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "574c1015-a63f-4160-8822-93abfc5af358", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
90162025-11-13T08:04:03.683ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = SupportBundles step id = Cleanup
90172025-11-13T08:04:03.683ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90182025-11-13T08:04:03.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90192025-11-13T08:04:03.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90202025-11-13T08:04:03.686ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
90212025-11-13T08:04:03.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90222025-11-13T08:04:03.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90232025-11-13T08:04:03.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90242025-11-13T08:04:03.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90252025-11-13T08:04:03.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90262025-11-13T08:04:03.691ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
90272025-11-13T08:04:03.695ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = OmicronZones step id = Cleanup
90282025-11-13T08:04:03.695ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
90292025-11-13T08:04:03.695ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
90302025-11-13T08:04:03.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90312025-11-13T08:04:03.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90322025-11-13T08:04:03.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90332025-11-13T08:04:03.698ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
90342025-11-13T08:04:03.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90352025-11-13T08:04:03.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90362025-11-13T08:04:03.699ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
90372025-11-13T08:04:03.699ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
90382025-11-13T08:04:03.699ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = Cockroach step id = Ensure
90392025-11-13T08:04:03.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
90402025-11-13T08:04:03.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
90412025-11-13T08:04:03.709ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
90422025-11-13T08:04:03.709ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = b73a435c-c8f6-49b4-92b1-c98995369b77 step component = MgsUpdates step id = Ensure
90432025-11-13T08:04:03.709ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 17 reason = Dependency
90442025-11-13T08:04:03.709ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Dependency
90452025-11-13T08:04:03.709ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90462025-11-13T08:04:03.709ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90472025-11-13T08:04:03.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90482025-11-13T08:04:03.709ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.101937353s iteration = 7 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4160,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":133850961,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":133850901,"secs":0}},"event_index":1,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":133868122,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36697291,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":36697231,"secs":0}},"event_index":2,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":170585803,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":122576580,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":122576529,"secs":0}},"event_index":3,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":293177443,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":290664195,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":290664155,"secs":0}},"event_index":4,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":583870938,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":386915746,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":386915696,"secs":0}},"event_index":5,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":970804925,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":68944775,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":68944745,"secs":0}},"event_index":6,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":39761860,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3491,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3461,"secs":0}},"event_index":7,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":39773981,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2370,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2340,"secs":0}},"event_index":8,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":39778721,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2780,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2740,"secs":0}},"event_index":9,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":39782871,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1610,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1570,"secs":0}},"event_index":10,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":39786071,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35362755,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":35362705,"secs":0}},"event_index":11,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":75149956,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":12672659,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":12672619,"secs":0}},"event_index":12,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":87832165,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4209092,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4209052,"secs":0}},"event_index":13,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":92050888,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":9581319,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":9581279,"secs":0}},"event_index":14,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":101639457,"secs":1}},{"data":{"attempt_elapsed":{"nanos":3780,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3740,"secs":0}},"event_index":15,"execution_id":"04f41689-5b79-4a82-b14c-9426c6ec5fa1","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":101647937,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
90492025-11-13T08:04:03.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90502025-11-13T08:04:03.711ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90512025-11-13T08:04:03.711ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90522025-11-13T08:04:03.714ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
90532025-11-13T08:04:03.714ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90542025-11-13T08:04:03.714ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90552025-11-13T08:04:03.714ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90562025-11-13T08:04:03.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90572025-11-13T08:04:03.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90582025-11-13T08:04:03.716ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90592025-11-13T08:04:03.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90602025-11-13T08:04:03.717ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(08c59e42-a65a-4a78-b24e-a61ef72ba1c2, 0), (25faec09-1a95-4211-b5bb-8965f08ec3c6, 0), (036cf7b1-53e4-423a-bec9-1e7a1c588e6a, 0), (94ba9306-3556-4350-a5d6-c083bbc60676, 0)] collection_id = 08c59e42-a65a-4a78-b24e-a61ef72ba1c2
90612025-11-13T08:04:03.717ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90622025-11-13T08:04:03.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90632025-11-13T08:04:03.717ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90642025-11-13T08:04:03.720ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
90652025-11-13T08:04:03.720ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
90662025-11-13T08:04:03.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
90672025-11-13T08:04:03.722ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
90682025-11-13T08:04:03.722ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.498936ms iteration = 17 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
90692025-11-13T08:04:03.847ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 08c59e42-a65a-4a78-b24e-a61ef72ba1c2 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
90702025-11-13T08:04:03.847ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90712025-11-13T08:04:03.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90722025-11-13T08:04:03.852ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(25faec09-1a95-4211-b5bb-8965f08ec3c6, 0), (036cf7b1-53e4-423a-bec9-1e7a1c588e6a, 0), (94ba9306-3556-4350-a5d6-c083bbc60676, 0), (7326b9b7-1c7b-4ff1-bfab-7fa1ed6db1b3, 0)] collection_id = 25faec09-1a95-4211-b5bb-8965f08ec3c6
90732025-11-13T08:04:03.852ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90742025-11-13T08:04:03.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90752025-11-13T08:04:03.966ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 25faec09-1a95-4211-b5bb-8965f08ec3c6 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
90762025-11-13T08:04:03.966ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90772025-11-13T08:04:03.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90782025-11-13T08:04:03.971ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(036cf7b1-53e4-423a-bec9-1e7a1c588e6a, 0), (94ba9306-3556-4350-a5d6-c083bbc60676, 0), (7326b9b7-1c7b-4ff1-bfab-7fa1ed6db1b3, 0), (6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7, 0)] collection_id = 036cf7b1-53e4-423a-bec9-1e7a1c588e6a
90792025-11-13T08:04:03.971ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90802025-11-13T08:04:03.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90812025-11-13T08:04:03.980ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "4d727b0b-7972-491e-b9fb-b07344d0e90e", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
90822025-11-13T08:04:03.980ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
90832025-11-13T08:04:03.980ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
90842025-11-13T08:04:03.980ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e1fc0c3-8597-4217-ba08-59b6fdce5b9d", "content-length": "682", "date": "Thu, 13 Nov 2025 08:04:03 GMT"} })
90852025-11-13T08:04:03.980ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90862025-11-13T08:04:03.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90872025-11-13T08:04:03.984ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90882025-11-13T08:04:03.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90892025-11-13T08:04:03.989ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90902025-11-13T08:04:03.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90912025-11-13T08:04:03.991ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
90922025-11-13T08:04:03.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
90932025-11-13T08:04:04.000ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.357068919s iteration = 11 status = {}
90942025-11-13T08:04:04.105ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 036cf7b1-53e4-423a-bec9-1e7a1c588e6a file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
90952025-11-13T08:04:04.105ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90962025-11-13T08:04:04.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
90972025-11-13T08:04:04.109ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(94ba9306-3556-4350-a5d6-c083bbc60676, 0), (7326b9b7-1c7b-4ff1-bfab-7fa1ed6db1b3, 0), (6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7, 0)]
90982025-11-13T08:04:04.110ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6252, tv_nsec: 462511120 } })
90992025-11-13T08:04:04.143ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6252, tv_nsec: 495674982 } })
91002025-11-13T08:04:04.143ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
91012025-11-13T08:04:04.143ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
91022025-11-13T08:04:04.143ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
91032025-11-13T08:04:04.144ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "82e49649-64d1-40ea-b12c-846d1466a8df", "content-length": "476", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91042025-11-13T08:04:04.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
91052025-11-13T08:04:04.144ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a70523e-8a08-4d00-b53c-14c23432e4de", "content-length": "734", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91062025-11-13T08:04:04.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
91072025-11-13T08:04:04.145ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c813ccd6-2c62-460b-bc18-3cd6b2d941ee", "content-length": "111", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91082025-11-13T08:04:04.145ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
91092025-11-13T08:04:04.145ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a401463-defd-4fb2-9ab3-365e8877904e", "content-length": "111", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91102025-11-13T08:04:04.145ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
91112025-11-13T08:04:04.146ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "309fc7ed-7a70-4328-b4b0-d1f1acb57c0b", "content-length": "163", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91122025-11-13T08:04:04.146ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
91132025-11-13T08:04:04.146ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1852c00e-0455-4746-b3d4-052e23d624f1", "content-length": "163", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91142025-11-13T08:04:04.146ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
91152025-11-13T08:04:04.146ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4991759d-9f03-48a1-9328-08238c6f3605", "content-length": "166", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91162025-11-13T08:04:04.146ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
91172025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "22707f34-7a17-466a-b794-306f6cfd6e96", "content-length": "166", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91182025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
91192025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e8ea51a-804e-4865-9620-eac7fb41d544", "content-length": "702", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91202025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
91212025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99057aa8-1b06-4252-ae56-71bab667d8ce", "content-length": "727", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91222025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
91232025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "07809233-5b9e-4075-8501-22aecd8931c8", "content-length": "729", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91242025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
91252025-11-13T08:04:04.147ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "24117ad8-9391-4750-b63a-7226160c2fd2", "content-length": "728", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91262025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
91272025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b546dc5-72db-4a63-9343-c3df713e51fb", "content-length": "722", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91282025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
91292025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "469fc891-9fd9-4633-9c7d-465ccc244219", "content-length": "10", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91302025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
91312025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8850109f-0b05-43a3-bc65-caa1d09cd817", "content-length": "144", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91322025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
91332025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcd26bab-4d46-4982-89ee-e953d325e30c", "content-length": "147", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91342025-11-13T08:04:04.148ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
91352025-11-13T08:04:04.149ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ed51b66-f4d4-41d6-9d12-c2206d26cefb", "content-length": "109", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91362025-11-13T08:04:04.149ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
91372025-11-13T08:04:04.149ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd4261ef-cbcf-4bbb-8ab6-9dbcf1dfa9db", "content-length": "109", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91382025-11-13T08:04:04.149ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
91392025-11-13T08:04:04.150ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5479582-5354-4eba-8128-3c709458ed54", "content-length": "163", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91402025-11-13T08:04:04.150ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
91412025-11-13T08:04:04.150ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2dda2c1-acaa-4b03-854a-046f52f52d64", "content-length": "163", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91422025-11-13T08:04:04.150ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
91432025-11-13T08:04:04.150ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "76a0b6c3-bdc6-44e2-9d37-a7c2b86b34f8", "content-length": "166", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91442025-11-13T08:04:04.150ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
91452025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "849d042f-f2ac-4560-86a7-23831ce4c34c", "content-length": "166", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91462025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
91472025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb223aa2-0e71-4dd6-9eeb-c5c8d0e1d24b", "content-length": "702", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91482025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
91492025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84ef7c19-7006-43c7-8fcc-5bd0e1eb93ba", "content-length": "727", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91502025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
91512025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "364188c0-93c1-43a2-8cb6-338faae9adb1", "content-length": "729", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91522025-11-13T08:04:04.151ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
91532025-11-13T08:04:04.152ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b0ff000-99a4-479b-91f1-ef7100038469", "content-length": "728", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91542025-11-13T08:04:04.152ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91552025-11-13T08:04:04.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91562025-11-13T08:04:04.153ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91572025-11-13T08:04:04.155ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
91582025-11-13T08:04:04.155ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91592025-11-13T08:04:04.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91602025-11-13T08:04:04.157ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91612025-11-13T08:04:04.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
91622025-11-13T08:04:04.159ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91632025-11-13T08:04:04.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
91642025-11-13T08:04:04.196ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
91652025-11-13T08:04:04.196ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
91662025-11-13T08:04:04.248ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c964678-52ba-459a-beec-73e8b6e9d2d0", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91672025-11-13T08:04:04.250ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
91682025-11-13T08:04:04.250ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
91692025-11-13T08:04:04.250ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
91702025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
91712025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
91722025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
91732025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
91742025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
91752025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4f47175-c72d-4fed-8657-3c620e38d22c", "content-length": "64", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91762025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
91772025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "26114788-9f1d-4593-9167-9a5f70616abe", "content-length": "64", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91782025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "70c49141-61c9-4869-b7f9-22fc3c4c49f7", "content-length": "64", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91792025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
91802025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
91812025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "22368a00-f4f5-45ad-839b-de0bfd42722c", "content-length": "64", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91822025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
91832025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c4d8f5a-bc58-49b5-b33f-3d3b6bfc3cc4", "content-length": "64", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91842025-11-13T08:04:04.250ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
91852025-11-13T08:04:04.288ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "4da25f45-45f7-4a27-9a28-d089e8e3f750", "content-length": "415188", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91862025-11-13T08:04:04.289ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "940bec9c-13f1-4223-9657-e00cc8724b93", "content-length": "387919", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91872025-11-13T08:04:04.290ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "24f966f4-b058-4819-b40d-65bf66db2882", "content-length": "397835", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91882025-11-13T08:04:04.290ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b80c198-a031-4312-8820-6f3fa2b2f980", "content-length": "390378", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91892025-11-13T08:04:04.291ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ecbabbd-7885-44d6-aec8-a8835835c57d", "content-length": "392736", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91902025-11-13T08:04:04.296ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
91912025-11-13T08:04:04.296ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
91922025-11-13T08:04:04.296ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
91932025-11-13T08:04:04.296ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
91942025-11-13T08:04:04.296ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
91952025-11-13T08:04:04.330ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
91962025-11-13T08:04:04.330ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
91972025-11-13T08:04:04.333ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "b29d79b8-e940-414d-afb7-bd890eb78e41", "content-length": "118", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
91982025-11-13T08:04:04.334ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
91992025-11-13T08:04:04.425ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
92002025-11-13T08:04:04.426ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
92012025-11-13T08:04:04.426ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "baf7178f-453d-4374-8201-99ae9b2f8250", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
92022025-11-13T08:04:04.426ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
92032025-11-13T08:04:04.428ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
92042025-11-13T08:04:04.428ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
92052025-11-13T08:04:04.428ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7912efe-4097-4fa2-8ff6-974cd1c5e05d", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
92062025-11-13T08:04:04.428ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
92072025-11-13T08:04:04.431ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
92082025-11-13T08:04:04.431ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
92092025-11-13T08:04:04.431ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b3e5f81-7cfd-4044-b590-13c6d354b0e9", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:04:04 GMT"} })
92102025-11-13T08:04:04.431ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
92112025-11-13T08:04:04.433ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
92122025-11-13T08:04:04.433ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
92132025-11-13T08:04:04.435ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92142025-11-13T08:04:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92152025-11-13T08:04:04.438ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92162025-11-13T08:04:04.440ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
92172025-11-13T08:04:04.440ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92182025-11-13T08:04:04.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
92192025-11-13T08:04:04.631ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a file = nexus/db-queries/src/db/datastore/inventory.rs:1613
92202025-11-13T08:04:04.631ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a time_started = 2025-11-13 08:04:04.143923 UTC
92212025-11-13T08:04:04.631ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 28 reason = Dependency
92222025-11-13T08:04:04.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
92232025-11-13T08:04:04.631ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 922.320021ms iteration = 7 status = {"collection_id":"a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a","time_done":"2025-11-13 08:04:04.433802 UTC","time_started":"2025-11-13 08:04:04.143923 UTC"}
92242025-11-13T08:04:04.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
92252025-11-13T08:04:04.633ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92262025-11-13T08:04:04.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
92272025-11-13T08:04:04.637ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
92282025-11-13T08:04:04.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
92292025-11-13T08:04:04.738ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
92302025-11-13T08:04:04.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
92312025-11-13T08:04:04.781ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 19 reason = Dependency
92322025-11-13T08:04:04.781ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 149.810085ms iteration = 28 status = {"Loaded":{"collection_id":"9ef70888-af19-4061-b38f-7f8e1ee65d5f","time_loaded":"2025-11-13T08:04:04.631886762Z","time_started":"2025-11-13T08:04:04.237104Z"}}
92332025-11-13T08:04:04.781ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 16 reason = Dependency
92342025-11-13T08:04:04.781ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 10 reason = Dependency
92352025-11-13T08:04:04.781ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92362025-11-13T08:04:04.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92372025-11-13T08:04:04.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92382025-11-13T08:04:04.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92392025-11-13T08:04:04.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92402025-11-13T08:04:04.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92412025-11-13T08:04:04.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92422025-11-13T08:04:04.783ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92432025-11-13T08:04:04.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92442025-11-13T08:04:04.786ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92452025-11-13T08:04:04.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92462025-11-13T08:04:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92472025-11-13T08:04:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
92482025-11-13T08:04:04.787ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92492025-11-13T08:04:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92502025-11-13T08:04:04.787ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92512025-11-13T08:04:04.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92522025-11-13T08:04:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92532025-11-13T08:04:04.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92542025-11-13T08:04:04.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92552025-11-13T08:04:04.789ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92562025-11-13T08:04:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92572025-11-13T08:04:04.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92582025-11-13T08:04:04.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92592025-11-13T08:04:04.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92602025-11-13T08:04:04.791ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92612025-11-13T08:04:04.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92622025-11-13T08:04:04.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92632025-11-13T08:04:04.793ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92642025-11-13T08:04:04.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92652025-11-13T08:04:04.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
92662025-11-13T08:04:04.793ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
92672025-11-13T08:04:04.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92682025-11-13T08:04:04.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92692025-11-13T08:04:04.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
92702025-11-13T08:04:04.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92712025-11-13T08:04:04.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92722025-11-13T08:04:04.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92732025-11-13T08:04:04.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92742025-11-13T08:04:04.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92752025-11-13T08:04:04.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92762025-11-13T08:04:04.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92772025-11-13T08:04:04.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92782025-11-13T08:04:04.801ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92792025-11-13T08:04:04.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92802025-11-13T08:04:04.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92812025-11-13T08:04:04.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92822025-11-13T08:04:04.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92832025-11-13T08:04:04.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92842025-11-13T08:04:04.803ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 21.735188ms iteration = 19 status = {"physical_disks_added":0}
92852025-11-13T08:04:04.804ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
92862025-11-13T08:04:04.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92872025-11-13T08:04:04.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92882025-11-13T08:04:04.805ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
92892025-11-13T08:04:04.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92902025-11-13T08:04:04.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92912025-11-13T08:04:04.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92922025-11-13T08:04:04.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92932025-11-13T08:04:04.807ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
92942025-11-13T08:04:04.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
92952025-11-13T08:04:04.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
92962025-11-13T08:04:04.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
92972025-11-13T08:04:04.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92982025-11-13T08:04:04.809ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
92992025-11-13T08:04:04.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93002025-11-13T08:04:04.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93012025-11-13T08:04:04.811ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
93022025-11-13T08:04:04.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
93032025-11-13T08:04:04.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
93042025-11-13T08:04:04.813ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
93052025-11-13T08:04:04.813ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 31.835296ms iteration = 10 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"9ef70888-af19-4061-b38f-7f8e1ee65d5f","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
93062025-11-13T08:04:04.814ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93072025-11-13T08:04:04.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93082025-11-13T08:04:04.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93092025-11-13T08:04:04.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93102025-11-13T08:04:04.816ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93112025-11-13T08:04:04.817ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93122025-11-13T08:04:04.820ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
93132025-11-13T08:04:04.820ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93142025-11-13T08:04:04.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93152025-11-13T08:04:04.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93162025-11-13T08:04:04.825ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93172025-11-13T08:04:04.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93182025-11-13T08:04:04.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93192025-11-13T08:04:04.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93202025-11-13T08:04:04.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93212025-11-13T08:04:04.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93222025-11-13T08:04:04.831ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
93232025-11-13T08:04:04.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93242025-11-13T08:04:04.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93252025-11-13T08:04:04.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93262025-11-13T08:04:04.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93272025-11-13T08:04:04.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93282025-11-13T08:04:04.836ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93292025-11-13T08:04:04.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93302025-11-13T08:04:04.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93312025-11-13T08:04:04.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93322025-11-13T08:04:04.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93332025-11-13T08:04:04.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93342025-11-13T08:04:04.842ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
93352025-11-13T08:04:04.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93362025-11-13T08:04:04.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93372025-11-13T08:04:04.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93382025-11-13T08:04:04.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93392025-11-13T08:04:04.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93402025-11-13T08:04:04.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93412025-11-13T08:04:04.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93422025-11-13T08:04:04.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93432025-11-13T08:04:04.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93442025-11-13T08:04:04.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93452025-11-13T08:04:04.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93462025-11-13T08:04:04.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93472025-11-13T08:04:04.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93482025-11-13T08:04:04.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93492025-11-13T08:04:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93502025-11-13T08:04:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93512025-11-13T08:04:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93522025-11-13T08:04:04.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93532025-11-13T08:04:04.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93542025-11-13T08:04:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93552025-11-13T08:04:04.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93562025-11-13T08:04:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93572025-11-13T08:04:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93582025-11-13T08:04:04.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
93592025-11-13T08:04:04.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93602025-11-13T08:04:04.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93612025-11-13T08:04:04.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93622025-11-13T08:04:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93632025-11-13T08:04:04.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93642025-11-13T08:04:04.885ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93652025-11-13T08:04:04.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93662025-11-13T08:04:04.889ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
93672025-11-13T08:04:04.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93682025-11-13T08:04:04.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93692025-11-13T08:04:04.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93702025-11-13T08:04:04.891ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93712025-11-13T08:04:04.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93722025-11-13T08:04:04.895ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
93732025-11-13T08:04:04.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93742025-11-13T08:04:04.896ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93752025-11-13T08:04:04.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93762025-11-13T08:04:04.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93772025-11-13T08:04:04.900ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93782025-11-13T08:04:04.903ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93792025-11-13T08:04:04.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93802025-11-13T08:04:04.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93812025-11-13T08:04:04.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93822025-11-13T08:04:04.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93832025-11-13T08:04:04.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93842025-11-13T08:04:04.909ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
93852025-11-13T08:04:04.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93862025-11-13T08:04:04.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93872025-11-13T08:04:04.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93882025-11-13T08:04:04.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93892025-11-13T08:04:04.912ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93902025-11-13T08:04:04.915ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
93912025-11-13T08:04:04.915ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93922025-11-13T08:04:04.915ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93932025-11-13T08:04:04.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93942025-11-13T08:04:04.916ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93952025-11-13T08:04:04.917ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93962025-11-13T08:04:04.920ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
93972025-11-13T08:04:04.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
93982025-11-13T08:04:04.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
93992025-11-13T08:04:04.962ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
94002025-11-13T08:04:04.962ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
94012025-11-13T08:04:04.962ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 8e7158c8-5b06-492d-95b1-6bfbf705d12f parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
94022025-11-13T08:04:04.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94032025-11-13T08:04:04.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94042025-11-13T08:04:04.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94052025-11-13T08:04:04.967ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
94062025-11-13T08:04:04.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
94072025-11-13T08:04:04.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
94082025-11-13T08:04:04.972ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
94092025-11-13T08:04:04.972ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
94102025-11-13T08:04:04.972ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 190.577567ms iteration = 16 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
94112025-11-13T08:04:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 274 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 8b95cc8c-b894-4c80-9019-16053e591a05 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
94122025-11-13T08:04:07.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 62 reason = Timeout
94132025-11-13T08:04:07.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94142025-11-13T08:04:07.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94152025-11-13T08:04:07.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94162025-11-13T08:04:07.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
94172025-11-13T08:04:07.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94182025-11-13T08:04:07.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94192025-11-13T08:04:07.626ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
94202025-11-13T08:04:07.626ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.60178ms iteration = 62 status = {"config_updated":false}
94212025-11-13T08:04:12.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 32 reason = Timeout
94222025-11-13T08:04:12.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94232025-11-13T08:04:12.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 63 reason = Timeout
94242025-11-13T08:04:12.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94252025-11-13T08:04:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94262025-11-13T08:04:12.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94272025-11-13T08:04:12.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94282025-11-13T08:04:12.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94292025-11-13T08:04:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
94302025-11-13T08:04:12.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94312025-11-13T08:04:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
94322025-11-13T08:04:12.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94332025-11-13T08:04:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94342025-11-13T08:04:12.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94352025-11-13T08:04:12.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
94362025-11-13T08:04:12.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.60437ms iteration = 63 status = {"config_updated":false}
94372025-11-13T08:04:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94382025-11-13T08:04:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94392025-11-13T08:04:12.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94402025-11-13T08:04:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
94412025-11-13T08:04:12.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94422025-11-13T08:04:12.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94432025-11-13T08:04:12.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 32 reason = Timeout
94442025-11-13T08:04:12.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94452025-11-13T08:04:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94462025-11-13T08:04:12.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
94472025-11-13T08:04:12.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
94482025-11-13T08:04:12.644ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
94492025-11-13T08:04:12.644ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.152994ms iteration = 32 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
94502025-11-13T08:04:12.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.440778ms iteration = 32 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
94512025-11-13T08:04:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 317 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 282c207d-f715-4d62-9b33-42a77e67ce70 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
94522025-11-13T08:04:17.617ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 29 reason = Timeout
94532025-11-13T08:04:17.617ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 64 reason = Timeout
94542025-11-13T08:04:17.617ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
94552025-11-13T08:04:17.617ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94562025-11-13T08:04:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
94572025-11-13T08:04:17.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94582025-11-13T08:04:17.619ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94592025-11-13T08:04:17.619ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94602025-11-13T08:04:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
94612025-11-13T08:04:17.624ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
94622025-11-13T08:04:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
94632025-11-13T08:04:17.624ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94642025-11-13T08:04:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
94652025-11-13T08:04:17.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94662025-11-13T08:04:17.625ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
94672025-11-13T08:04:17.625ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.947846ms iteration = 64 status = {"config_updated":false}
94682025-11-13T08:04:17.626ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9ef70888-af19-4061-b38f-7f8e1ee65d5f original_time_started = 2025-11-13 08:04:04.237104 UTC
94692025-11-13T08:04:17.626ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.319024ms iteration = 29 status = {"Loaded":{"collection_id":"9ef70888-af19-4061-b38f-7f8e1ee65d5f","time_loaded":"2025-11-13T08:04:17.617852254Z","time_started":"2025-11-13T08:04:04.237104Z"}}
94702025-11-13T08:04:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 22 reason = Timeout
94712025-11-13T08:04:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 22 reason = Timeout
94722025-11-13T08:04:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94732025-11-13T08:04:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
94742025-11-13T08:04:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94752025-11-13T08:04:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
94762025-11-13T08:04:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94772025-11-13T08:04:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94782025-11-13T08:04:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94792025-11-13T08:04:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94802025-11-13T08:04:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
94812025-11-13T08:04:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
94822025-11-13T08:04:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
94832025-11-13T08:04:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
94842025-11-13T08:04:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.206482ms iteration = 22 status = "NoSitrep"
94852025-11-13T08:04:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94862025-11-13T08:04:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94872025-11-13T08:04:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
94882025-11-13T08:04:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
94892025-11-13T08:04:17.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94902025-11-13T08:04:17.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
94912025-11-13T08:04:17.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
94922025-11-13T08:04:17.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.997696ms iteration = 22 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
94932025-11-13T08:04:22.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 33 reason = Timeout
94942025-11-13T08:04:22.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 65 reason = Timeout
94952025-11-13T08:04:22.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
94962025-11-13T08:04:22.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
94972025-11-13T08:04:22.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
94982025-11-13T08:04:22.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
94992025-11-13T08:04:22.621ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95002025-11-13T08:04:22.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95012025-11-13T08:04:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
95022025-11-13T08:04:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95032025-11-13T08:04:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95042025-11-13T08:04:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95052025-11-13T08:04:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95062025-11-13T08:04:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95072025-11-13T08:04:22.627ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
95082025-11-13T08:04:22.627ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.071108ms iteration = 65 status = {"config_updated":false}
95092025-11-13T08:04:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95102025-11-13T08:04:22.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95112025-11-13T08:04:22.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95122025-11-13T08:04:22.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
95132025-11-13T08:04:22.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95142025-11-13T08:04:22.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95152025-11-13T08:04:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 33 reason = Timeout
95162025-11-13T08:04:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95172025-11-13T08:04:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95182025-11-13T08:04:22.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
95192025-11-13T08:04:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
95202025-11-13T08:04:22.646ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.910027ms iteration = 33 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
95212025-11-13T08:04:22.651ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
95222025-11-13T08:04:22.651ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.465198ms iteration = 33 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
95232025-11-13T08:04:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 282 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = f4c59177-016b-4b3e-ac2f-5204bdc3a524 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
95242025-11-13T08:04:27.618ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 66 reason = Timeout
95252025-11-13T08:04:27.618ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95262025-11-13T08:04:27.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95272025-11-13T08:04:27.620ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95282025-11-13T08:04:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
95292025-11-13T08:04:27.623ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95302025-11-13T08:04:27.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95312025-11-13T08:04:27.624ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
95322025-11-13T08:04:27.624ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.434766ms iteration = 66 status = {"config_updated":false}
95332025-11-13T08:04:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 12 reason = Timeout
95342025-11-13T08:04:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 12 reason = Timeout
95352025-11-13T08:04:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 12 reason = Timeout
95362025-11-13T08:04:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
95372025-11-13T08:04:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
95382025-11-13T08:04:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
95392025-11-13T08:04:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
95402025-11-13T08:04:32.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6280, tv_nsec: 962994076 } })
95412025-11-13T08:04:32.610ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6280, tv_nsec: 963048407 } })
95422025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 18.140957ms iteration = 12 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
95432025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 18 reason = Timeout
95442025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95452025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 20 reason = Timeout
95462025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 30 reason = Timeout
95472025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 34 reason = Timeout
95482025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95492025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
95502025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95512025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 67 reason = Timeout
95522025-11-13T08:04:32.625ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
95532025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95542025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
95552025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95562025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95572025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
95582025-11-13T08:04:32.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95592025-11-13T08:04:32.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95602025-11-13T08:04:32.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95612025-11-13T08:04:32.628ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95622025-11-13T08:04:32.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95632025-11-13T08:04:32.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95642025-11-13T08:04:32.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
95652025-11-13T08:04:32.633ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95662025-11-13T08:04:32.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95672025-11-13T08:04:32.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
95682025-11-13T08:04:32.633ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
95692025-11-13T08:04:32.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
95702025-11-13T08:04:32.633ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95712025-11-13T08:04:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
95722025-11-13T08:04:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
95732025-11-13T08:04:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
95742025-11-13T08:04:32.634ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
95752025-11-13T08:04:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
95762025-11-13T08:04:32.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95772025-11-13T08:04:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
95782025-11-13T08:04:32.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95792025-11-13T08:04:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95802025-11-13T08:04:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95812025-11-13T08:04:32.636ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9ef70888-af19-4061-b38f-7f8e1ee65d5f original_time_started = 2025-11-13 08:04:04.237104 UTC
95822025-11-13T08:04:32.636ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 11.443795ms iteration = 30 status = {"Loaded":{"collection_id":"9ef70888-af19-4061-b38f-7f8e1ee65d5f","time_loaded":"2025-11-13T08:04:32.625312959Z","time_started":"2025-11-13T08:04:04.237104Z"}}
95832025-11-13T08:04:32.636ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
95842025-11-13T08:04:32.636ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.410424ms iteration = 67 status = {"config_updated":false}
95852025-11-13T08:04:32.637ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
95862025-11-13T08:04:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
95872025-11-13T08:04:32.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
95882025-11-13T08:04:32.637ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95892025-11-13T08:04:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
95902025-11-13T08:04:32.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95912025-11-13T08:04:32.639ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95922025-11-13T08:04:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
95932025-11-13T08:04:32.641ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
95942025-11-13T08:04:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
95952025-11-13T08:04:32.641ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 12 reason = Timeout
95962025-11-13T08:04:32.641ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
95972025-11-13T08:04:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
95982025-11-13T08:04:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 12 reason = Timeout
95992025-11-13T08:04:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96002025-11-13T08:04:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96012025-11-13T08:04:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 12 reason = Timeout
96022025-11-13T08:04:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
96032025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
96042025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 12 reason = Timeout
96052025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
96062025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96072025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
96082025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 12 reason = Timeout
96092025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
96102025-11-13T08:04:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
96112025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 18 reason = Timeout
96122025-11-13T08:04:32.644ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
96132025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96142025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96152025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 12 reason = Timeout
96162025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96172025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96182025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 34 reason = Timeout
96192025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96202025-11-13T08:04:32.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96212025-11-13T08:04:32.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
96222025-11-13T08:04:32.646ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
96232025-11-13T08:04:32.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
96242025-11-13T08:04:32.647ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
96252025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
96262025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 12 reason = Timeout
96272025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 12 reason = Timeout
96282025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
96292025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
96302025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 23 reason = Timeout
96312025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 12 reason = Timeout
96322025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
96332025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
96342025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 23 reason = Timeout
96352025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96362025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96372025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 12 reason = Timeout
96382025-11-13T08:04:32.648ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
96392025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
96402025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96412025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 12 reason = Timeout
96422025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
96432025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 12 reason = Timeout
96442025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
96452025-11-13T08:04:32.648ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96462025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
96472025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
96482025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
96492025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96502025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
96512025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96522025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
96532025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
96542025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
96552025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
96562025-11-13T08:04:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6281, tv_nsec: 1763838 } })
96572025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
96582025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
96592025-11-13T08:04:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96602025-11-13T08:04:32.650ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96612025-11-13T08:04:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
96622025-11-13T08:04:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96632025-11-13T08:04:32.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96642025-11-13T08:04:32.652ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96652025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 4.56271ms iteration = 12 status = {"errors":[],"finish_invoked_ok":[]}
96662025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 4.787944ms iteration = 12 status = {"errors":[],"garbage_collect_requested":[]}
96672025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
96682025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
96692025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
96702025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
96712025-11-13T08:04:32.653ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
96722025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
96732025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
96742025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96752025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
96762025-11-13T08:04:32.653ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96772025-11-13T08:04:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
96782025-11-13T08:04:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
96792025-11-13T08:04:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
96802025-11-13T08:04:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96812025-11-13T08:04:32.655ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96822025-11-13T08:04:32.655ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
96832025-11-13T08:04:32.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
96842025-11-13T08:04:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
96852025-11-13T08:04:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
96862025-11-13T08:04:32.657ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6281, tv_nsec: 9885798 } })
96872025-11-13T08:04:32.658ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
96882025-11-13T08:04:32.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.062708ms iteration = 23 status = "NoSitrep"
96892025-11-13T08:04:32.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
96902025-11-13T08:04:32.658ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
96912025-11-13T08:04:32.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
96922025-11-13T08:04:32.659ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 34.215843ms iteration = 20 status = {"physical_disks_added":0}
96932025-11-13T08:04:32.659ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 10.887883ms iteration = 12 status = {"errors":[],"requests_created_ok":[]}
96942025-11-13T08:04:32.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
96952025-11-13T08:04:32.659ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
96962025-11-13T08:04:32.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
96972025-11-13T08:04:32.660ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
96982025-11-13T08:04:32.660ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
96992025-11-13T08:04:32.660ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 18.84623ms iteration = 12 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
97002025-11-13T08:04:32.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
97012025-11-13T08:04:32.661ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
97022025-11-13T08:04:32.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
97032025-11-13T08:04:32.662ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 19.633855ms iteration = 12 status = 0
97042025-11-13T08:04:32.665ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97052025-11-13T08:04:32.665ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97062025-11-13T08:04:32.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97072025-11-13T08:04:32.665ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
97082025-11-13T08:04:32.667ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
97092025-11-13T08:04:32.667ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 19.096225ms iteration = 23 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
97102025-11-13T08:04:32.679ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97112025-11-13T08:04:32.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97122025-11-13T08:04:32.681ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97132025-11-13T08:04:32.683ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
97142025-11-13T08:04:32.683ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 57.782376ms iteration = 34 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
97152025-11-13T08:04:32.683ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
97162025-11-13T08:04:32.685ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 78.114036ms iteration = 12 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
97172025-11-13T08:04:32.685ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97182025-11-13T08:04:32.685ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97192025-11-13T08:04:32.685ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97202025-11-13T08:04:32.687ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97212025-11-13T08:04:32.691ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
97222025-11-13T08:04:32.691ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
97232025-11-13T08:04:32.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
97242025-11-13T08:04:32.709ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97252025-11-13T08:04:32.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97262025-11-13T08:04:32.713ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
97272025-11-13T08:04:32.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
97282025-11-13T08:04:32.714ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
97292025-11-13T08:04:32.714ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 88.745385ms iteration = 18 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
97302025-11-13T08:04:32.716ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
97312025-11-13T08:04:32.716ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
97322025-11-13T08:04:32.716ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 73.571096ms iteration = 12 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
97332025-11-13T08:04:32.717ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 69.117189ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
97342025-11-13T08:04:32.718ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
97352025-11-13T08:04:32.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
97362025-11-13T08:04:32.719ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.035995ms iteration = 34 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
97372025-11-13T08:04:32.725ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 82.555023ms iteration = 12 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
97382025-11-13T08:04:32.758ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
97392025-11-13T08:04:32.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
97402025-11-13T08:04:32.760ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
97412025-11-13T08:04:32.760ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
97422025-11-13T08:04:32.761ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a618e1ae-0ca9-4f58-8dcd-d9186f0d318d", "content-length": "26", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97432025-11-13T08:04:32.761ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
97442025-11-13T08:04:32.761ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 112.794208ms iteration = 12 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
97452025-11-13T08:04:32.763ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
97462025-11-13T08:04:32.796ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
97472025-11-13T08:04:32.796ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dbbcf9e-df43-48e2-b9bf-ba8d8e1f60cc", "content-length": "78", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97482025-11-13T08:04:32.796ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
97492025-11-13T08:04:32.796ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
97502025-11-13T08:04:32.796ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
97512025-11-13T08:04:32.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
97522025-11-13T08:04:32.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
97532025-11-13T08:04:32.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
97542025-11-13T08:04:32.799ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
97552025-11-13T08:04:32.799ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
97562025-11-13T08:04:32.799ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
97572025-11-13T08:04:32.802ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
97582025-11-13T08:04:32.802ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
97592025-11-13T08:04:32.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
97602025-11-13T08:04:32.803ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
97612025-11-13T08:04:32.803ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
97622025-11-13T08:04:32.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
97632025-11-13T08:04:32.804ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
97642025-11-13T08:04:32.804ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
97652025-11-13T08:04:32.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
97662025-11-13T08:04:32.805ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
97672025-11-13T08:04:32.805ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
97682025-11-13T08:04:32.805ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1efd232-fb0a-4129-8297-fd082b057cd6", "content-length": "81", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97692025-11-13T08:04:32.806ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
97702025-11-13T08:04:32.806ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
97712025-11-13T08:04:32.806ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "cba0ce18-4708-4224-9409-6907942e7818", "content-length": "81", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97722025-11-13T08:04:32.807ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
97732025-11-13T08:04:32.807ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
97742025-11-13T08:04:32.807ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ec39272-b8b9-4899-816b-1e85f55df03f", "content-length": "81", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97752025-11-13T08:04:32.809ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 160.618378ms iteration = 12 status = {"disabled":false,"errors":[],"sps":[]}
97762025-11-13T08:04:32.870ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
97772025-11-13T08:04:32.870ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
97782025-11-13T08:04:32.871ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "db012f4b-7fb3-4572-98dc-26899e2ce546", "content-length": "26", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97792025-11-13T08:04:32.871ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
97802025-11-13T08:04:32.872ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
97812025-11-13T08:04:32.900ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
97822025-11-13T08:04:32.903ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3657bf4-eed9-4c89-9350-89e233fa16f6", "content-length": "691", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97832025-11-13T08:04:32.903ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
97842025-11-13T08:04:32.903ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
97852025-11-13T08:04:32.903ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
97862025-11-13T08:04:32.904ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 261.548112ms iteration = 12 status = {}
97872025-11-13T08:04:32.974ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 367.182089ms iteration = 12 status = {}
97882025-11-13T08:04:32.977ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6281, tv_nsec: 329762977 } })
97892025-11-13T08:04:32.978ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
97902025-11-13T08:04:32.978ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
97912025-11-13T08:04:32.982ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "f10bee02-475e-454b-b04e-a23cdcbcbc4f", "content-length": "372", "date": "Thu, 13 Nov 2025 08:04:32 GMT"} })
97922025-11-13T08:04:32.982ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97932025-11-13T08:04:32.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97942025-11-13T08:04:33.031ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
97952025-11-13T08:04:33.031ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
97962025-11-13T08:04:33.032ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c6340e01-5730-4aa0-880e-c72ea3d53400", "date": "Thu, 13 Nov 2025 08:04:33 GMT"} })
97972025-11-13T08:04:33.032ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
97982025-11-13T08:04:33.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
97992025-11-13T08:04:33.034ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98002025-11-13T08:04:33.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98012025-11-13T08:04:33.037ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
98022025-11-13T08:04:33.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
98032025-11-13T08:04:33.041ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
98042025-11-13T08:04:33.041ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
98052025-11-13T08:04:33.042ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98062025-11-13T08:04:33.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98072025-11-13T08:04:33.042ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "fd1aa32c-38ad-473a-bd73-5d986324f8b3", "date": "Thu, 13 Nov 2025 08:04:33 GMT"} })
98082025-11-13T08:04:33.044ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
98092025-11-13T08:04:33.044ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98102025-11-13T08:04:33.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98112025-11-13T08:04:33.045ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 401.504374ms iteration = 18 status = {}
98122025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
98132025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
98142025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
98152025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
98162025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98172025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98182025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98192025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
98202025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
98212025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
98222025-11-13T08:04:33.054ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
98232025-11-13T08:04:33.055ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98242025-11-13T08:04:33.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98252025-11-13T08:04:33.058ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
98262025-11-13T08:04:33.058ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
98272025-11-13T08:04:33.199ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d5bd0036-0500-499c-aa5b-41a4f997573a", "date": "Thu, 13 Nov 2025 08:04:33 GMT"} })
98282025-11-13T08:04:33.199ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
98292025-11-13T08:04:33.199ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
98302025-11-13T08:04:33.200ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0a796c6-49b7-4f3c-ae52-7276e76b114e", "content-length": "682", "date": "Thu, 13 Nov 2025 08:04:33 GMT"} })
98312025-11-13T08:04:33.200ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98322025-11-13T08:04:33.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98332025-11-13T08:04:33.203ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98342025-11-13T08:04:33.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98352025-11-13T08:04:33.207ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98362025-11-13T08:04:33.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98372025-11-13T08:04:33.209ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
98382025-11-13T08:04:33.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
98392025-11-13T08:04:33.216ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 572.476944ms iteration = 12 status = {}
98402025-11-13T08:04:35.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 483 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = a3fe2d16-a281-4539-b2b2-3d3a54f027ef response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
98412025-11-13T08:04:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 68 reason = Timeout
98422025-11-13T08:04:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98432025-11-13T08:04:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98442025-11-13T08:04:37.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98452025-11-13T08:04:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
98462025-11-13T08:04:37.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98472025-11-13T08:04:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98482025-11-13T08:04:37.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
98492025-11-13T08:04:37.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.299304ms iteration = 68 status = {"config_updated":false}
98502025-11-13T08:04:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 35 reason = Timeout
98512025-11-13T08:04:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98522025-11-13T08:04:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 69 reason = Timeout
98532025-11-13T08:04:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98542025-11-13T08:04:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98552025-11-13T08:04:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98562025-11-13T08:04:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98572025-11-13T08:04:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98582025-11-13T08:04:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
98592025-11-13T08:04:42.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98602025-11-13T08:04:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98612025-11-13T08:04:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
98622025-11-13T08:04:42.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98632025-11-13T08:04:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98642025-11-13T08:04:42.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98652025-11-13T08:04:42.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98662025-11-13T08:04:42.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
98672025-11-13T08:04:42.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.843234ms iteration = 69 status = {"config_updated":false}
98682025-11-13T08:04:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98692025-11-13T08:04:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
98702025-11-13T08:04:42.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
98712025-11-13T08:04:42.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
98722025-11-13T08:04:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 35 reason = Timeout
98732025-11-13T08:04:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98742025-11-13T08:04:42.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98752025-11-13T08:04:42.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
98762025-11-13T08:04:42.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
98772025-11-13T08:04:42.646ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.525049ms iteration = 35 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
98782025-11-13T08:04:42.651ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
98792025-11-13T08:04:42.651ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.95155ms iteration = 35 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
98802025-11-13T08:04:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 276 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 515584ca-feca-4f41-9f3b-2b484313c076 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
98812025-11-13T08:04:47.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 31 reason = Timeout
98822025-11-13T08:04:47.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
98832025-11-13T08:04:47.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 70 reason = Timeout
98842025-11-13T08:04:47.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98852025-11-13T08:04:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
98862025-11-13T08:04:47.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98872025-11-13T08:04:47.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98882025-11-13T08:04:47.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98892025-11-13T08:04:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
98902025-11-13T08:04:47.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
98912025-11-13T08:04:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
98922025-11-13T08:04:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
98932025-11-13T08:04:47.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
98942025-11-13T08:04:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
98952025-11-13T08:04:47.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
98962025-11-13T08:04:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.827294ms iteration = 70 status = {"config_updated":false}
98972025-11-13T08:04:47.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9ef70888-af19-4061-b38f-7f8e1ee65d5f original_time_started = 2025-11-13 08:04:04.237104 UTC
98982025-11-13T08:04:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.013528ms iteration = 31 status = {"Loaded":{"collection_id":"9ef70888-af19-4061-b38f-7f8e1ee65d5f","time_loaded":"2025-11-13T08:04:47.626441077Z","time_started":"2025-11-13T08:04:04.237104Z"}}
98992025-11-13T08:04:47.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 24 reason = Timeout
99002025-11-13T08:04:47.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
99012025-11-13T08:04:47.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 24 reason = Timeout
99022025-11-13T08:04:47.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
99032025-11-13T08:04:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
99042025-11-13T08:04:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
99052025-11-13T08:04:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
99062025-11-13T08:04:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
99072025-11-13T08:04:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99082025-11-13T08:04:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99092025-11-13T08:04:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
99102025-11-13T08:04:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
99112025-11-13T08:04:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
99122025-11-13T08:04:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
99132025-11-13T08:04:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.193992ms iteration = 24 status = "NoSitrep"
99142025-11-13T08:04:47.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
99152025-11-13T08:04:47.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
99162025-11-13T08:04:47.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
99172025-11-13T08:04:47.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
99182025-11-13T08:04:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99192025-11-13T08:04:47.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
99202025-11-13T08:04:47.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
99212025-11-13T08:04:47.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.032707ms iteration = 24 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
99222025-11-13T08:04:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 36 reason = Timeout
99232025-11-13T08:04:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 71 reason = Timeout
99242025-11-13T08:04:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99252025-11-13T08:04:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99262025-11-13T08:04:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99272025-11-13T08:04:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99282025-11-13T08:04:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99292025-11-13T08:04:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99302025-11-13T08:04:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
99312025-11-13T08:04:52.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99322025-11-13T08:04:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
99332025-11-13T08:04:52.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99342025-11-13T08:04:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99352025-11-13T08:04:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99362025-11-13T08:04:52.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99372025-11-13T08:04:52.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
99382025-11-13T08:04:52.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.981767ms iteration = 71 status = {"config_updated":false}
99392025-11-13T08:04:52.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99402025-11-13T08:04:52.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99412025-11-13T08:04:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
99422025-11-13T08:04:52.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
99432025-11-13T08:04:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
99442025-11-13T08:04:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 36 reason = Timeout
99452025-11-13T08:04:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99462025-11-13T08:04:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99472025-11-13T08:04:52.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
99482025-11-13T08:04:52.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
99492025-11-13T08:04:52.650ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
99502025-11-13T08:04:52.650ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.252546ms iteration = 36 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
99512025-11-13T08:04:52.685ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 43.155248ms iteration = 36 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
99522025-11-13T08:04:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 325 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 643cf0e1-464a-4c7d-b841-9db5cf35f62b response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
99532025-11-13T08:04:57.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 72 reason = Timeout
99542025-11-13T08:04:57.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99552025-11-13T08:04:57.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99562025-11-13T08:04:57.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99572025-11-13T08:04:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
99582025-11-13T08:04:57.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
99592025-11-13T08:04:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
99602025-11-13T08:04:57.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
99612025-11-13T08:04:57.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.401716ms iteration = 72 status = {"config_updated":false}
99622025-11-13T08:05:00.472ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:58555
99632025-11-13T08:05:00.472ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 8c0a7084-43d7-4a90-a1dc-779dc616ba6c uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
99642025-11-13T08:05:00.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 8c0a7084-43d7-4a90-a1dc-779dc616ba6c roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
99652025-11-13T08:05:00.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 8c0a7084-43d7-4a90-a1dc-779dc616ba6c resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
99662025-11-13T08:05:00.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2557 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 8c0a7084-43d7-4a90-a1dc-779dc616ba6c response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
99672025-11-13T08:05:00.475ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = c94a6bbd-d2b7-436e-aa44-c0573d4bb8ee uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
99682025-11-13T08:05:00.475ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = c94a6bbd-d2b7-436e-aa44-c0573d4bb8ee roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
99692025-11-13T08:05:00.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = c94a6bbd-d2b7-436e-aa44-c0573d4bb8ee resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
99702025-11-13T08:05:00.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2932 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = c94a6bbd-d2b7-436e-aa44-c0573d4bb8ee response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
99712025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 7 reason = Timeout
99722025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 7 reason = Timeout
99732025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 7 reason = Timeout
99742025-11-13T08:05:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:55:02.607112196 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
99752025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
99762025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
99772025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 7 reason = Timeout
99782025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 7 reason = Timeout
99792025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 9 reason = Timeout
99802025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
99812025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 7 reason = Timeout
99822025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
99832025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
99842025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 13 reason = Timeout
99852025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
99862025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 13 reason = Timeout
99872025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 13 reason = Timeout
99882025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
99892025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
99902025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 9 reason = Timeout
99912025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 8 reason = Timeout
99922025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
99932025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
99942025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
99952025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
99962025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
99972025-11-13T08:05:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
99982025-11-13T08:05:02.607ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = ExternalNetworking step id = Ensure
99992025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
100002025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100012025-11-13T08:05:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100022025-11-13T08:05:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6310, tv_nsec: 960355279 } })
100032025-11-13T08:05:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6310, tv_nsec: 960353579 } })
100042025-11-13T08:05:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100052025-11-13T08:05:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
100062025-11-13T08:05:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.673463ms iteration = 7 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
100072025-11-13T08:05:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
100082025-11-13T08:05:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.656383ms iteration = 7 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
100092025-11-13T08:05:02.608ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100102025-11-13T08:05:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
100112025-11-13T08:05:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100122025-11-13T08:05:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
100132025-11-13T08:05:02.611ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:55:02.607112196 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
100142025-11-13T08:05:02.611ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.372206ms iteration = 7 status = {"expiration":"2025-11-13T07:55:02.607112196Z","pruned":[]}
100152025-11-13T08:05:02.611ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100162025-11-13T08:05:02.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100172025-11-13T08:05:02.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100182025-11-13T08:05:02.614ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6310, tv_nsec: 966819806 } })
100192025-11-13T08:05:02.614ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6310, tv_nsec: 966853817 } })
100202025-11-13T08:05:02.615ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 8.416645ms iteration = 13 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
100212025-11-13T08:05:02.615ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
100222025-11-13T08:05:02.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100232025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100242025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
100252025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100262025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 4 reason = Timeout
100272025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 7 reason = Timeout
100282025-11-13T08:05:02.617ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
100292025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
100302025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100312025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
100322025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 17 reason = Timeout
100332025-11-13T08:05:02.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100342025-11-13T08:05:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100352025-11-13T08:05:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
100362025-11-13T08:05:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100372025-11-13T08:05:02.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100382025-11-13T08:05:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100392025-11-13T08:05:02.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100402025-11-13T08:05:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100412025-11-13T08:05:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100422025-11-13T08:05:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
100432025-11-13T08:05:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100442025-11-13T08:05:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100452025-11-13T08:05:02.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
100462025-11-13T08:05:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100472025-11-13T08:05:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
100482025-11-13T08:05:02.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100492025-11-13T08:05:02.621ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
100502025-11-13T08:05:02.621ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
100512025-11-13T08:05:02.621ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
100522025-11-13T08:05:02.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100532025-11-13T08:05:02.621ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100542025-11-13T08:05:02.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100552025-11-13T08:05:02.621ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100562025-11-13T08:05:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
100572025-11-13T08:05:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
100582025-11-13T08:05:02.622ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100592025-11-13T08:05:02.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100602025-11-13T08:05:02.623ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100612025-11-13T08:05:02.624ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100622025-11-13T08:05:02.625ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
100632025-11-13T08:05:02.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
100642025-11-13T08:05:02.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
100652025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
100662025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
100672025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
100682025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 21 reason = Timeout
100692025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 32 reason = Timeout
100702025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
100712025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
100722025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 37 reason = Timeout
100732025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
100742025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 19 reason = Timeout
100752025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
100762025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 73 reason = Timeout
100772025-11-13T08:05:02.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
100782025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
100792025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
100802025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
100812025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
100822025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
100832025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
100842025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
100852025-11-13T08:05:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
100862025-11-13T08:05:02.628ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100872025-11-13T08:05:02.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100882025-11-13T08:05:02.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100892025-11-13T08:05:02.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100902025-11-13T08:05:02.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100912025-11-13T08:05:02.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100922025-11-13T08:05:02.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100932025-11-13T08:05:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
100942025-11-13T08:05:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
100952025-11-13T08:05:02.632ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
100962025-11-13T08:05:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
100972025-11-13T08:05:02.632ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
100982025-11-13T08:05:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
100992025-11-13T08:05:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
101002025-11-13T08:05:02.634ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
101012025-11-13T08:05:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
101022025-11-13T08:05:02.634ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
101032025-11-13T08:05:02.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
101042025-11-13T08:05:02.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
101052025-11-13T08:05:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
101062025-11-13T08:05:02.634ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101072025-11-13T08:05:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
101082025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
101092025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
101102025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101112025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
101122025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101132025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101142025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101152025-11-13T08:05:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101162025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
101172025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
101182025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
101192025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
101202025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101212025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
101222025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101232025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
101242025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
101252025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
101262025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
101272025-11-13T08:05:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
101282025-11-13T08:05:02.637ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101292025-11-13T08:05:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
101302025-11-13T08:05:02.637ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
101312025-11-13T08:05:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
101322025-11-13T08:05:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
101332025-11-13T08:05:02.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101342025-11-13T08:05:02.637ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
101352025-11-13T08:05:02.638ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101362025-11-13T08:05:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
101372025-11-13T08:05:02.638ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
101382025-11-13T08:05:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
101392025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101402025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 9ef70888-af19-4061-b38f-7f8e1ee65d5f original_time_started = 2025-11-13 08:04:04.237104 UTC
101412025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.639168ms iteration = 32 status = {"Loaded":{"collection_id":"9ef70888-af19-4061-b38f-7f8e1ee65d5f","time_loaded":"2025-11-13T08:05:02.626671121Z","time_started":"2025-11-13T08:04:04.237104Z"}}
101422025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
101432025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
101442025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
101452025-11-13T08:05:02.639ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
101462025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
101472025-11-13T08:05:02.639ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101482025-11-13T08:05:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
101492025-11-13T08:05:02.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
101502025-11-13T08:05:02.641ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
101512025-11-13T08:05:02.641ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.441615ms iteration = 73 status = {"config_updated":false}
101522025-11-13T08:05:02.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
101532025-11-13T08:05:02.641ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101542025-11-13T08:05:02.641ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
101552025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 7 reason = Timeout
101562025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 13 reason = Timeout
101572025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
101582025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
101592025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 19 reason = Timeout
101602025-11-13T08:05:02.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
101612025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
101622025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 13 reason = Timeout
101632025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
101642025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 37 reason = Timeout
101652025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
101662025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101672025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 13 reason = Timeout
101682025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
101692025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 13 reason = Timeout
101702025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 13 reason = Timeout
101712025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
101722025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
101732025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
101742025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 13 reason = Timeout
101752025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 7 reason = Timeout
101762025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
101772025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
101782025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
101792025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
101802025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
101812025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
101822025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
101832025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
101842025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
101852025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
101862025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
101872025-11-13T08:05:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
101882025-11-13T08:05:02.643ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
101892025-11-13T08:05:02.643ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
101902025-11-13T08:05:02.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101912025-11-13T08:05:02.643ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
101922025-11-13T08:05:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
101932025-11-13T08:05:02.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101942025-11-13T08:05:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
101952025-11-13T08:05:02.644ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
101962025-11-13T08:05:02.644ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101972025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101982025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
101992025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102002025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
102012025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
102022025-11-13T08:05:02.645ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
102032025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
102042025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
102052025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 7 reason = Timeout
102062025-11-13T08:05:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
102072025-11-13T08:05:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102082025-11-13T08:05:02.646ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
102092025-11-13T08:05:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
102102025-11-13T08:05:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
102112025-11-13T08:05:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
102122025-11-13T08:05:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
102132025-11-13T08:05:02.647ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
102142025-11-13T08:05:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
102152025-11-13T08:05:02.649ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
102162025-11-13T08:05:02.649ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102172025-11-13T08:05:02.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 13 reason = Timeout
102182025-11-13T08:05:02.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 13 reason = Timeout
102192025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
102202025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
102212025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
102222025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102232025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
102242025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 7 reason = Timeout
102252025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
102262025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102272025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 13 reason = Timeout
102282025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
102292025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
102302025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 13 reason = Timeout
102312025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 25 reason = Timeout
102322025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
102332025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
102342025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 7 reason = Timeout
102352025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
102362025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
102372025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 13 reason = Timeout
102382025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
102392025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
102402025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
102412025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
102422025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 25 reason = Timeout
102432025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
102442025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
102452025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
102462025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 13 reason = Timeout
102472025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
102482025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 7 reason = Timeout
102492025-11-13T08:05:02.650ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
102502025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
102512025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
102522025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
102532025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
102542025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
102552025-11-13T08:05:02.650ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6311, tv_nsec: 2843394 } })
102562025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
102572025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
102582025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
102592025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
102602025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
102612025-11-13T08:05:02.650ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
102622025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 43.75807ms iteration = 7 status = {"generation":2}
102632025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
102642025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
102652025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
102662025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
102672025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
102682025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
102692025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
102702025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
102712025-11-13T08:05:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
102722025-11-13T08:05:02.652ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
102732025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
102742025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
102752025-11-13T08:05:02.654ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
102762025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
102772025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 6.330685ms iteration = 13 status = {"errors":[],"finish_invoked_ok":[]}
102782025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
102792025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
102802025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
102812025-11-13T08:05:02.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
102822025-11-13T08:05:02.655ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
102832025-11-13T08:05:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
102842025-11-13T08:05:02.655ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 9.301813ms iteration = 7 status = {"errors":[],"found_port_ok":[]}
102852025-11-13T08:05:02.657ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
102862025-11-13T08:05:02.658ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
102872025-11-13T08:05:02.658ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
102882025-11-13T08:05:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
102892025-11-13T08:05:02.658ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
102902025-11-13T08:05:02.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
102912025-11-13T08:05:02.661ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
102922025-11-13T08:05:02.661ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 12.851522ms iteration = 7 status = {"by_rx":{},"error":null}
102932025-11-13T08:05:02.661ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
102942025-11-13T08:05:02.669ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
102952025-11-13T08:05:02.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
102962025-11-13T08:05:02.672ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102972025-11-13T08:05:02.673ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102982025-11-13T08:05:02.677ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102992025-11-13T08:05:02.677ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
103002025-11-13T08:05:02.677ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
103012025-11-13T08:05:02.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
103022025-11-13T08:05:02.678ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
103032025-11-13T08:05:02.678ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
103042025-11-13T08:05:02.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
103052025-11-13T08:05:02.684ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
103062025-11-13T08:05:02.684ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
103072025-11-13T08:05:02.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
103082025-11-13T08:05:02.686ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
103092025-11-13T08:05:02.689ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
103102025-11-13T08:05:02.690ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = SledList step id = Fetch
103112025-11-13T08:05:02.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
103122025-11-13T08:05:02.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
103132025-11-13T08:05:02.691ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
103142025-11-13T08:05:02.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
103152025-11-13T08:05:02.695ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
103162025-11-13T08:05:02.695ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 53.700015ms iteration = 13 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
103172025-11-13T08:05:02.696ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 46.998734ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
103182025-11-13T08:05:02.698ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 56.147294ms iteration = 37 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
103192025-11-13T08:05:02.700ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6311, tv_nsec: 52462110 } })
103202025-11-13T08:05:02.700ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 58.444529ms iteration = 13 status = 0
103212025-11-13T08:05:02.700ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
103222025-11-13T08:05:02.702ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103232025-11-13T08:05:02.702ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103242025-11-13T08:05:02.704ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 62.703083ms iteration = 13 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
103252025-11-13T08:05:02.704ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
103262025-11-13T08:05:02.705ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103272025-11-13T08:05:02.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
103282025-11-13T08:05:02.705ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 56.836467ms iteration = 25 status = "NoSitrep"
103292025-11-13T08:05:02.706ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103302025-11-13T08:05:02.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103312025-11-13T08:05:02.709ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
103322025-11-13T08:05:02.709ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 60.669763ms iteration = 7 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
103332025-11-13T08:05:02.710ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
103342025-11-13T08:05:02.710ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
103352025-11-13T08:05:02.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
103362025-11-13T08:05:02.711ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
103372025-11-13T08:05:02.711ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
103382025-11-13T08:05:02.711ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
103392025-11-13T08:05:02.711ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
103402025-11-13T08:05:02.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
103412025-11-13T08:05:02.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
103422025-11-13T08:05:02.712ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
103432025-11-13T08:05:02.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
103442025-11-13T08:05:02.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
103452025-11-13T08:05:02.713ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 96.157901ms iteration = 4 status = {"nsuccess":5}
103462025-11-13T08:05:02.716ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
103472025-11-13T08:05:02.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
103482025-11-13T08:05:02.722ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103492025-11-13T08:05:02.722ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
103502025-11-13T08:05:02.722ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
103512025-11-13T08:05:02.722ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
103522025-11-13T08:05:02.722ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 115.695425ms iteration = 7 status = {"generation":1}
103532025-11-13T08:05:02.724ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
103542025-11-13T08:05:02.724ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 97.567258ms iteration = 37 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
103552025-11-13T08:05:02.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103562025-11-13T08:05:02.725ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
103572025-11-13T08:05:02.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
103582025-11-13T08:05:02.726ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
103592025-11-13T08:05:02.726ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
103602025-11-13T08:05:02.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
103612025-11-13T08:05:02.726ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
103622025-11-13T08:05:02.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
103632025-11-13T08:05:02.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
103642025-11-13T08:05:02.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
103652025-11-13T08:05:02.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
103662025-11-13T08:05:02.744ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103672025-11-13T08:05:02.745ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = DeployNexusRecords step id = Ensure
103682025-11-13T08:05:02.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
103692025-11-13T08:05:02.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
103702025-11-13T08:05:02.746ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
103712025-11-13T08:05:02.747ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
103722025-11-13T08:05:02.748ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 98.65953ms iteration = 25 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
103732025-11-13T08:05:02.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103742025-11-13T08:05:02.752ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
103752025-11-13T08:05:02.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
103762025-11-13T08:05:02.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
103772025-11-13T08:05:02.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
103782025-11-13T08:05:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
103792025-11-13T08:05:02.761ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
103802025-11-13T08:05:02.762ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 135.28602ms iteration = 19 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
103812025-11-13T08:05:02.762ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 113.714166ms iteration = 13 status = {"errors":[],"requests_created_ok":[]}
103822025-11-13T08:05:02.763ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 115.627553ms iteration = 13 status = {"errors":[],"garbage_collect_requested":[]}
103832025-11-13T08:05:02.767ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
103842025-11-13T08:05:02.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
103852025-11-13T08:05:02.769ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
103862025-11-13T08:05:02.769ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
103872025-11-13T08:05:02.769ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 127.361375ms iteration = 7 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
103882025-11-13T08:05:02.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103892025-11-13T08:05:02.770ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
103902025-11-13T08:05:02.770ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
103912025-11-13T08:05:02.770ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 153.179192ms iteration = 7 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
103922025-11-13T08:05:02.775ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
103932025-11-13T08:05:02.778ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
103942025-11-13T08:05:02.778ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
103952025-11-13T08:05:02.778ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 136.062155ms iteration = 13 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
103962025-11-13T08:05:02.778ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
103972025-11-13T08:05:02.778ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 171.636925ms iteration = 7 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
103982025-11-13T08:05:02.780ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = SledAgent step id = Ensure
103992025-11-13T08:05:02.833ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104002025-11-13T08:05:02.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104012025-11-13T08:05:02.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104022025-11-13T08:05:02.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
104032025-11-13T08:05:02.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104042025-11-13T08:05:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104052025-11-13T08:05:02.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104062025-11-13T08:05:02.843ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
104072025-11-13T08:05:02.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
104082025-11-13T08:05:02.845ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 196.82759ms iteration = 13 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
104092025-11-13T08:05:02.846ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
104102025-11-13T08:05:02.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104112025-11-13T08:05:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104122025-11-13T08:05:02.880ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
104132025-11-13T08:05:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
104142025-11-13T08:05:02.902ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104152025-11-13T08:05:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
104162025-11-13T08:05:02.913ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104172025-11-13T08:05:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
104182025-11-13T08:05:02.916ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104192025-11-13T08:05:02.920ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
104202025-11-13T08:05:02.920ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104212025-11-13T08:05:02.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
104222025-11-13T08:05:02.923ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104232025-11-13T08:05:02.924ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 297.581361ms iteration = 21 status = {"physical_disks_added":0}
104242025-11-13T08:05:02.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104252025-11-13T08:05:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104262025-11-13T08:05:02.927ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
104272025-11-13T08:05:02.927ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
104282025-11-13T08:05:02.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104292025-11-13T08:05:02.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
104302025-11-13T08:05:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104312025-11-13T08:05:02.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104322025-11-13T08:05:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104332025-11-13T08:05:02.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104342025-11-13T08:05:02.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104352025-11-13T08:05:02.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104362025-11-13T08:05:02.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104372025-11-13T08:05:02.944ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
104382025-11-13T08:05:02.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104392025-11-13T08:05:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
104402025-11-13T08:05:02.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104412025-11-13T08:05:02.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104422025-11-13T08:05:02.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104432025-11-13T08:05:02.954ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104442025-11-13T08:05:02.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104452025-11-13T08:05:02.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104462025-11-13T08:05:02.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104472025-11-13T08:05:02.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104482025-11-13T08:05:02.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104492025-11-13T08:05:02.964ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
104502025-11-13T08:05:02.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104512025-11-13T08:05:02.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104522025-11-13T08:05:02.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104532025-11-13T08:05:02.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104542025-11-13T08:05:02.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104552025-11-13T08:05:02.973ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
104562025-11-13T08:05:02.973ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104572025-11-13T08:05:02.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104582025-11-13T08:05:02.975ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
104592025-11-13T08:05:02.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104602025-11-13T08:05:02.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104612025-11-13T08:05:02.977ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 369.77974ms iteration = 13 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
104622025-11-13T08:05:02.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104632025-11-13T08:05:02.984ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
104642025-11-13T08:05:02.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104652025-11-13T08:05:02.984ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104662025-11-13T08:05:03.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104672025-11-13T08:05:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104682025-11-13T08:05:03.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104692025-11-13T08:05:03.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104702025-11-13T08:05:03.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104712025-11-13T08:05:03.015ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
104722025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "a237106f-4d1e-45b1-b39d-fc7f328bfa8e", "content-length": "78", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
104732025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
104742025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
104752025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104762025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
104772025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
104782025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
104792025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
104802025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
104812025-11-13T08:05:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
104822025-11-13T08:05:03.020ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
104832025-11-13T08:05:03.020ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
104842025-11-13T08:05:03.020ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
104852025-11-13T08:05:03.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104862025-11-13T08:05:03.024ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
104872025-11-13T08:05:03.024ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
104882025-11-13T08:05:03.024ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
104892025-11-13T08:05:03.024ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
104902025-11-13T08:05:03.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
104912025-11-13T08:05:03.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
104922025-11-13T08:05:03.026ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
104932025-11-13T08:05:03.026ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
104942025-11-13T08:05:03.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
104952025-11-13T08:05:03.027ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
104962025-11-13T08:05:03.027ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
104972025-11-13T08:05:03.028ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac9c52a2-8a01-439c-8843-e7e1ca620af9", "content-length": "81", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
104982025-11-13T08:05:03.028ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
104992025-11-13T08:05:03.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105002025-11-13T08:05:03.029ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
105012025-11-13T08:05:03.029ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
105022025-11-13T08:05:03.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105032025-11-13T08:05:03.029ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "65a67b8b-c8b0-4b42-abc4-042a188c662b", "content-length": "81", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
105042025-11-13T08:05:03.031ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
105052025-11-13T08:05:03.031ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
105062025-11-13T08:05:03.032ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "97983841-e7fc-49f1-bf1c-511663fdb98e", "content-length": "81", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
105072025-11-13T08:05:03.032ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
105082025-11-13T08:05:03.033ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105092025-11-13T08:05:03.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105102025-11-13T08:05:03.033ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 385.149552ms iteration = 13 status = {"disabled":false,"errors":[],"sps":[]}
105112025-11-13T08:05:03.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105122025-11-13T08:05:03.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105132025-11-13T08:05:03.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105142025-11-13T08:05:03.041ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
105152025-11-13T08:05:03.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105162025-11-13T08:05:03.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105172025-11-13T08:05:03.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105182025-11-13T08:05:03.049ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
105192025-11-13T08:05:03.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105202025-11-13T08:05:03.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105212025-11-13T08:05:03.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105222025-11-13T08:05:03.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105232025-11-13T08:05:03.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105242025-11-13T08:05:03.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105252025-11-13T08:05:03.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105262025-11-13T08:05:03.058ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
105272025-11-13T08:05:03.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105282025-11-13T08:05:03.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105292025-11-13T08:05:03.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105302025-11-13T08:05:03.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105312025-11-13T08:05:03.060ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
105322025-11-13T08:05:03.060ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 418.722792ms iteration = 7 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
105332025-11-13T08:05:03.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105342025-11-13T08:05:03.065ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
105352025-11-13T08:05:03.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105362025-11-13T08:05:03.067ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
105372025-11-13T08:05:03.067ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
105382025-11-13T08:05:03.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105392025-11-13T08:05:03.067ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4710455-90fe-47da-a3e5-e2a022225254", "content-length": "26", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
105402025-11-13T08:05:03.067ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
105412025-11-13T08:05:03.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105422025-11-13T08:05:03.070ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
105432025-11-13T08:05:03.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105442025-11-13T08:05:03.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105452025-11-13T08:05:03.072ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
105462025-11-13T08:05:03.072ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
105472025-11-13T08:05:03.073ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d31f1fa6-cbe5-491f-901d-c967bca714c2", "content-length": "26", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
105482025-11-13T08:05:03.073ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
105492025-11-13T08:05:03.075ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
105502025-11-13T08:05:03.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
105512025-11-13T08:05:03.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105522025-11-13T08:05:03.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105532025-11-13T08:05:03.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105542025-11-13T08:05:03.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105552025-11-13T08:05:03.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105562025-11-13T08:05:03.085ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
105572025-11-13T08:05:03.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105582025-11-13T08:05:03.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105592025-11-13T08:05:03.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105602025-11-13T08:05:03.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105612025-11-13T08:05:03.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105622025-11-13T08:05:03.095ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
105632025-11-13T08:05:03.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105642025-11-13T08:05:03.095ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105652025-11-13T08:05:03.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105662025-11-13T08:05:03.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105672025-11-13T08:05:03.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105682025-11-13T08:05:03.102ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
105692025-11-13T08:05:03.102ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105702025-11-13T08:05:03.102ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105712025-11-13T08:05:03.103ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
105722025-11-13T08:05:03.103ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
105732025-11-13T08:05:03.104ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = f7fb142b-0b22-45d2-8ad9-3ac71cacbe92 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
105742025-11-13T08:05:03.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105752025-11-13T08:05:03.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105762025-11-13T08:05:03.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105772025-11-13T08:05:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
105782025-11-13T08:05:03.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
105792025-11-13T08:05:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
105802025-11-13T08:05:03.112ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
105812025-11-13T08:05:03.112ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
105822025-11-13T08:05:03.112ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 495.122494ms iteration = 17 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
105832025-11-13T08:05:03.123ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
105842025-11-13T08:05:03.123ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "164f39cf-43ef-4dba-9fe5-df59e0ee19b8", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
105852025-11-13T08:05:03.124ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
105862025-11-13T08:05:03.125ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
105872025-11-13T08:05:03.125ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 476.099881ms iteration = 7 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
105882025-11-13T08:05:03.158ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
105892025-11-13T08:05:03.158ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
105902025-11-13T08:05:03.159ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "04f83588-8aeb-4187-96c8-e064a2fd2302", "content-length": "372", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
105912025-11-13T08:05:03.159ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105922025-11-13T08:05:03.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105932025-11-13T08:05:03.178ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
105942025-11-13T08:05:03.178ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
105952025-11-13T08:05:03.178ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9724096c-8bab-4078-94e9-f181e0c407a0", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
105962025-11-13T08:05:03.178ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105972025-11-13T08:05:03.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
105982025-11-13T08:05:03.180ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
105992025-11-13T08:05:03.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106002025-11-13T08:05:03.183ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
106012025-11-13T08:05:03.183ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7e9a580-99cc-408c-a101-97805029cd1b", "content-length": "691", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
106022025-11-13T08:05:03.183ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
106032025-11-13T08:05:03.183ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
106042025-11-13T08:05:03.183ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
106052025-11-13T08:05:03.184ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106062025-11-13T08:05:03.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106072025-11-13T08:05:03.185ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 543.396173ms iteration = 13 status = {}
106082025-11-13T08:05:03.187ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
106092025-11-13T08:05:03.187ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
106102025-11-13T08:05:03.188ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "abae28a6-303a-4b2b-99c1-7f9c50954bd2", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
106112025-11-13T08:05:03.190ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 548.022694ms iteration = 19 status = {}
106122025-11-13T08:05:03.190ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 20 reason = Dependency
106132025-11-13T08:05:03.190ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
106142025-11-13T08:05:03.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106152025-11-13T08:05:03.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106162025-11-13T08:05:03.192ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106172025-11-13T08:05:03.194ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
106182025-11-13T08:05:03.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106192025-11-13T08:05:03.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106202025-11-13T08:05:03.195ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106212025-11-13T08:05:03.198ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
106222025-11-13T08:05:03.198ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
106232025-11-13T08:05:03.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
106242025-11-13T08:05:03.236ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
106252025-11-13T08:05:03.237ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "cc29d592-738d-41be-bc56-87bf6d3167b4", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
106262025-11-13T08:05:03.239ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = FirewallRules step id = Ensure
106272025-11-13T08:05:03.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106282025-11-13T08:05:03.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106292025-11-13T08:05:03.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106302025-11-13T08:05:03.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106312025-11-13T08:05:03.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106322025-11-13T08:05:03.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106332025-11-13T08:05:03.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106342025-11-13T08:05:03.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106352025-11-13T08:05:03.246ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106362025-11-13T08:05:03.246ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106372025-11-13T08:05:03.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106382025-11-13T08:05:03.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106392025-11-13T08:05:03.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106402025-11-13T08:05:03.251ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
106412025-11-13T08:05:03.254ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106422025-11-13T08:05:03.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106432025-11-13T08:05:03.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106442025-11-13T08:05:03.255ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106452025-11-13T08:05:03.255ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106462025-11-13T08:05:03.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106472025-11-13T08:05:03.257ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106482025-11-13T08:05:03.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106492025-11-13T08:05:03.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106502025-11-13T08:05:03.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106512025-11-13T08:05:03.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106522025-11-13T08:05:03.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106532025-11-13T08:05:03.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106542025-11-13T08:05:03.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106552025-11-13T08:05:03.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106562025-11-13T08:05:03.268ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106572025-11-13T08:05:03.268ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106582025-11-13T08:05:03.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106592025-11-13T08:05:03.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106602025-11-13T08:05:03.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106612025-11-13T08:05:03.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106622025-11-13T08:05:03.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106632025-11-13T08:05:03.273ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106642025-11-13T08:05:03.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106652025-11-13T08:05:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
106662025-11-13T08:05:03.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106672025-11-13T08:05:03.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106682025-11-13T08:05:03.282ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106692025-11-13T08:05:03.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106702025-11-13T08:05:03.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106712025-11-13T08:05:03.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106722025-11-13T08:05:03.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106732025-11-13T08:05:03.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106742025-11-13T08:05:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106752025-11-13T08:05:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106762025-11-13T08:05:03.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106772025-11-13T08:05:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106782025-11-13T08:05:03.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106792025-11-13T08:05:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106802025-11-13T08:05:03.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106812025-11-13T08:05:03.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
106822025-11-13T08:05:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106832025-11-13T08:05:03.297ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106842025-11-13T08:05:03.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106852025-11-13T08:05:03.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106862025-11-13T08:05:03.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106872025-11-13T08:05:03.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106882025-11-13T08:05:03.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106892025-11-13T08:05:03.308ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
106902025-11-13T08:05:03.308ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106912025-11-13T08:05:03.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106922025-11-13T08:05:03.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106932025-11-13T08:05:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106942025-11-13T08:05:03.313ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106952025-11-13T08:05:03.314ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106962025-11-13T08:05:03.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
106972025-11-13T08:05:03.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
106982025-11-13T08:05:03.317ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6311, tv_nsec: 669448560 } })
106992025-11-13T08:05:03.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107002025-11-13T08:05:03.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107012025-11-13T08:05:03.319ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107022025-11-13T08:05:03.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107032025-11-13T08:05:03.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107042025-11-13T08:05:03.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107052025-11-13T08:05:03.323ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107062025-11-13T08:05:03.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107072025-11-13T08:05:03.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107082025-11-13T08:05:03.328ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
107092025-11-13T08:05:03.328ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3c307a82-942d-494c-acdf-ffa8a9e00555", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
107102025-11-13T08:05:03.328ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
107112025-11-13T08:05:03.330ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
107122025-11-13T08:05:03.330ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107132025-11-13T08:05:03.330ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107142025-11-13T08:05:03.330ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "92a2ef41-736d-47cb-8844-4b5f16e25a43", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
107152025-11-13T08:05:03.330ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
107162025-11-13T08:05:03.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107172025-11-13T08:05:03.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107182025-11-13T08:05:03.333ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
107192025-11-13T08:05:03.333ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 725.894301ms iteration = 9 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
107202025-11-13T08:05:03.333ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107212025-11-13T08:05:03.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107222025-11-13T08:05:03.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107232025-11-13T08:05:03.340ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
107242025-11-13T08:05:03.340ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107252025-11-13T08:05:03.340ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107262025-11-13T08:05:03.347ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
107272025-11-13T08:05:03.347ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107282025-11-13T08:05:03.348ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107292025-11-13T08:05:03.349ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107302025-11-13T08:05:03.351ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
107312025-11-13T08:05:03.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107322025-11-13T08:05:03.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107332025-11-13T08:05:03.353ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
107342025-11-13T08:05:03.353ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
107352025-11-13T08:05:03.353ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
107362025-11-13T08:05:03.385ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
107372025-11-13T08:05:03.387ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "7b726f01-a0e0-4122-a813-3639f8dcad72", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
107382025-11-13T08:05:03.387ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
107392025-11-13T08:05:03.389ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
107402025-11-13T08:05:03.389ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
107412025-11-13T08:05:03.389ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "af505656-2618-41a4-b0cf-ea66abac53e5", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
107422025-11-13T08:05:03.389ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
107432025-11-13T08:05:03.390ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e87bf0e-3065-4aa0-b3ee-52d1a11adbde", "content-length": "372", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
107442025-11-13T08:05:03.390ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107452025-11-13T08:05:03.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107462025-11-13T08:05:03.391ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "bb5c3b3b-3579-4ac2-8fe5-45950c6becbf", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
107472025-11-13T08:05:03.391ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
107482025-11-13T08:05:03.394ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
107492025-11-13T08:05:03.394ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 787.074004ms iteration = 9 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
107502025-11-13T08:05:03.397ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 789.840228ms iteration = 13 status = {}
107512025-11-13T08:05:03.405ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
107522025-11-13T08:05:03.405ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
107532025-11-13T08:05:03.406ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "50bf6826-7c20-4505-bcad-594edffb0e8a", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
107542025-11-13T08:05:03.406ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107552025-11-13T08:05:03.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107562025-11-13T08:05:03.408ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107572025-11-13T08:05:03.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107582025-11-13T08:05:03.411ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
107592025-11-13T08:05:03.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
107602025-11-13T08:05:03.414ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
107612025-11-13T08:05:03.414ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
107622025-11-13T08:05:03.415ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "79ed792d-d8c4-46c6-bb00-53c6d6b4d1f8", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
107632025-11-13T08:05:03.416ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 226.749427ms iteration = 20 status = {}
107642025-11-13T08:05:03.417ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
107652025-11-13T08:05:03.417ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
107662025-11-13T08:05:03.453ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107672025-11-13T08:05:03.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107682025-11-13T08:05:03.456ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
107692025-11-13T08:05:03.456ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107702025-11-13T08:05:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107712025-11-13T08:05:03.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107722025-11-13T08:05:03.456ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107732025-11-13T08:05:03.458ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107742025-11-13T08:05:03.459ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107752025-11-13T08:05:03.460ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107762025-11-13T08:05:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
107772025-11-13T08:05:03.464ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
107782025-11-13T08:05:03.467ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "77467c95-5e35-49ba-9d61-97f49c68290e", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
107792025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
107802025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
107812025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
107822025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
107832025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
107842025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
107852025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
107862025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
107872025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
107882025-11-13T08:05:03.469ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
107892025-11-13T08:05:03.470ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
107902025-11-13T08:05:03.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
107912025-11-13T08:05:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
107922025-11-13T08:05:03.470ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
107932025-11-13T08:05:03.470ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = Dns step id = Ensure
107942025-11-13T08:05:03.470ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
107952025-11-13T08:05:03.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107962025-11-13T08:05:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107972025-11-13T08:05:03.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
107982025-11-13T08:05:03.471ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
107992025-11-13T08:05:03.472ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
108002025-11-13T08:05:03.472ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
108012025-11-13T08:05:03.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108022025-11-13T08:05:03.476ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108032025-11-13T08:05:03.477ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
108042025-11-13T08:05:03.477ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
108052025-11-13T08:05:03.477ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
108062025-11-13T08:05:03.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108072025-11-13T08:05:03.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108082025-11-13T08:05:03.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108092025-11-13T08:05:03.481ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108102025-11-13T08:05:03.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108112025-11-13T08:05:03.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108122025-11-13T08:05:03.483ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
108132025-11-13T08:05:03.483ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
108142025-11-13T08:05:03.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108152025-11-13T08:05:03.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108162025-11-13T08:05:03.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108172025-11-13T08:05:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108182025-11-13T08:05:03.488ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108192025-11-13T08:05:03.488ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108202025-11-13T08:05:03.493ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
108212025-11-13T08:05:03.493ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
108222025-11-13T08:05:03.493ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
108232025-11-13T08:05:03.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108242025-11-13T08:05:03.493ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108252025-11-13T08:05:03.494ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108262025-11-13T08:05:03.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108272025-11-13T08:05:03.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108282025-11-13T08:05:03.499ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108292025-11-13T08:05:03.500ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
108302025-11-13T08:05:03.500ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
108312025-11-13T08:05:03.500ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
108322025-11-13T08:05:03.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108332025-11-13T08:05:03.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108342025-11-13T08:05:03.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108352025-11-13T08:05:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108362025-11-13T08:05:03.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108372025-11-13T08:05:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108382025-11-13T08:05:03.505ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 2
108392025-11-13T08:05:03.505ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
108402025-11-13T08:05:03.507ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
108412025-11-13T08:05:03.507ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
108422025-11-13T08:05:03.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108432025-11-13T08:05:03.507ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108442025-11-13T08:05:03.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108452025-11-13T08:05:03.513ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108462025-11-13T08:05:03.513ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108472025-11-13T08:05:03.513ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108482025-11-13T08:05:03.515ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
108492025-11-13T08:05:03.515ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
108502025-11-13T08:05:03.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108512025-11-13T08:05:03.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108522025-11-13T08:05:03.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108532025-11-13T08:05:03.519ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
108542025-11-13T08:05:03.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108552025-11-13T08:05:03.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108562025-11-13T08:05:03.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108572025-11-13T08:05:03.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108582025-11-13T08:05:03.523ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108592025-11-13T08:05:03.523ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108602025-11-13T08:05:03.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108612025-11-13T08:05:03.527ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
108622025-11-13T08:05:03.529ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
108632025-11-13T08:05:03.529ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
108642025-11-13T08:05:03.529ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = OmicronZones step id = Cleanup
108652025-11-13T08:05:03.529ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = OmicronZones step id = Cleanup
108662025-11-13T08:05:03.529ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = PhysicalDisks step id = Cleanup
108672025-11-13T08:05:03.529ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = Clickhouse step id = Ensure
108682025-11-13T08:05:03.529ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = Clickhouse step id = Ensure
108692025-11-13T08:05:03.544ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
108702025-11-13T08:05:03.544ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:52880
108712025-11-13T08:05:03.544ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52880 req_id = 08f67873-9a69-46ad-8494-124bc4488942 uri = /metrics/producers version = 1.0.0
108722025-11-13T08:05:03.544ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52880 req_id = 08f67873-9a69-46ad-8494-124bc4488942 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
108732025-11-13T08:05:03.544ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52880 req_id = 08f67873-9a69-46ad-8494-124bc4488942 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
108742025-11-13T08:05:03.567ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
108752025-11-13T08:05:03.572ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "ab2182b4-b8a7-455a-aff2-baa9215cc6e0", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
108762025-11-13T08:05:03.574ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = SupportBundles step id = Cleanup
108772025-11-13T08:05:03.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108782025-11-13T08:05:03.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108792025-11-13T08:05:03.575ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108802025-11-13T08:05:03.577ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
108812025-11-13T08:05:03.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108822025-11-13T08:05:03.578ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108832025-11-13T08:05:03.579ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108842025-11-13T08:05:03.579ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108852025-11-13T08:05:03.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108862025-11-13T08:05:03.583ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
108872025-11-13T08:05:03.588ZINFOnexus: assigned collector to new producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d file = nexus/src/app/oximeter.rs:76 producer_id = bcb58129-79cc-406c-aa16-455a5cd9f031
108882025-11-13T08:05:03.588ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 44385 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:52880 req_id = 08f67873-9a69-46ad-8494-124bc4488942 response_code = 201 uri = /metrics/producers
108892025-11-13T08:05:03.588ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = OmicronZones step id = Cleanup
108902025-11-13T08:05:03.588ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
108912025-11-13T08:05:03.588ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
108922025-11-13T08:05:03.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108932025-11-13T08:05:03.588ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "08f67873-9a69-46ad-8494-124bc4488942", "content-length": "41", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
108942025-11-13T08:05:03.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
108952025-11-13T08:05:03.590ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
108962025-11-13T08:05:03.590ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108972025-11-13T08:05:03.591ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
108982025-11-13T08:05:03.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
108992025-11-13T08:05:03.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
109002025-11-13T08:05:03.626ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
109012025-11-13T08:05:03.626ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
109022025-11-13T08:05:03.626ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = Cockroach step id = Ensure
109032025-11-13T08:05:03.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
109042025-11-13T08:05:03.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
109052025-11-13T08:05:03.634ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
109062025-11-13T08:05:03.634ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c0070224-245d-4cd9-a93b-f19397b62411 step component = MgsUpdates step id = Ensure
109072025-11-13T08:05:03.635ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Dependency
109082025-11-13T08:05:03.635ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 20 reason = Dependency
109092025-11-13T08:05:03.635ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109102025-11-13T08:05:03.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
109112025-11-13T08:05:03.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
109122025-11-13T08:05:03.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109132025-11-13T08:05:03.635ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.027517461s iteration = 8 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3270,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":82313959,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":82313909,"secs":0}},"event_index":1,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":82328659,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":55637484,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":55637434,"secs":0}},"event_index":2,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":137984323,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":34267854,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":34267804,"secs":0}},"event_index":3,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":172280187,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":459425133,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":459425083,"secs":0}},"event_index":4,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":631719940,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":230769877,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":230769837,"secs":0}},"event_index":5,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":862502197,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":59061061,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":59061021,"secs":0}},"event_index":6,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":921579888,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5090,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5050,"secs":0}},"event_index":7,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":921597959,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2330,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2290,"secs":0}},"event_index":8,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":921602959,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2920,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2890,"secs":0}},"event_index":9,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":921607539,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1830,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1800,"secs":0}},"event_index":10,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":921611269,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":44632628,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":44632588,"secs":0}},"event_index":11,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":966248967,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":14483185,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":14483145,"secs":0}},"event_index":12,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":980743301,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":38186381,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":38186311,"secs":0}},"event_index":13,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":18936262,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":8102680,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":8102630,"secs":0}},"event_index":14,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":27051572,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5590,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5550,"secs":0}},"event_index":15,"execution_id":"9f0926ae-5bca-4424-8660-32cf285e8c05","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":27065812,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
109142025-11-13T08:05:03.638ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109152025-11-13T08:05:03.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109162025-11-13T08:05:03.641ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
109172025-11-13T08:05:03.641ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109182025-11-13T08:05:03.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
109192025-11-13T08:05:03.641ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
109202025-11-13T08:05:03.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109212025-11-13T08:05:03.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
109222025-11-13T08:05:03.644ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
109232025-11-13T08:05:03.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
109242025-11-13T08:05:03.645ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109252025-11-13T08:05:03.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
109262025-11-13T08:05:03.648ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
109272025-11-13T08:05:03.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
109282025-11-13T08:05:03.650ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
109292025-11-13T08:05:03.650ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 15.037856ms iteration = 20 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
109302025-11-13T08:05:03.692ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "45246c11-2314-4d6a-865b-0d3e8eb28765", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
109312025-11-13T08:05:03.692ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
109322025-11-13T08:05:03.692ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
109332025-11-13T08:05:03.693ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "78f2f4f2-df81-4840-a9dd-1ff764ea74af", "content-length": "682", "date": "Thu, 13 Nov 2025 08:05:03 GMT"} })
109342025-11-13T08:05:03.693ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109352025-11-13T08:05:03.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109362025-11-13T08:05:03.696ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109372025-11-13T08:05:03.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109382025-11-13T08:05:03.700ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109392025-11-13T08:05:03.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109402025-11-13T08:05:03.702ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
109412025-11-13T08:05:03.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
109422025-11-13T08:05:03.709ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.067760852s iteration = 13 status = {}
109432025-11-13T08:05:03.766ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(7326b9b7-1c7b-4ff1-bfab-7fa1ed6db1b3, 0), (6bf3cea9-19be-40ec-ab5f-0ebf7a999bb7, 0), (a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a, 0), (64225556-e244-4e8f-b528-cc1f9cf4dea6, 0)] collection_id = 7326b9b7-1c7b-4ff1-bfab-7fa1ed6db1b3
109442025-11-13T08:05:03.766ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109452025-11-13T08:05:03.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109462025-11-13T08:05:03.947ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 7326b9b7-1c7b-4ff1-bfab-7fa1ed6db1b3 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
109472025-11-13T08:05:03.947ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109482025-11-13T08:05:03.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
109492025-11-13T08:05:04.022ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a, 0), (64225556-e244-4e8f-b528-cc1f9cf4dea6, 0), (9ef70888-af19-4061-b38f-7f8e1ee65d5f, 0)]
109502025-11-13T08:05:04.022ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6312, tv_nsec: 375256416 } })
109512025-11-13T08:05:04.055ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6312, tv_nsec: 407353367 } })
109522025-11-13T08:05:04.055ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
109532025-11-13T08:05:04.055ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
109542025-11-13T08:05:04.055ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
109552025-11-13T08:05:04.056ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d334518d-b5f9-4118-9e06-3fe02a57edee", "content-length": "476", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109562025-11-13T08:05:04.056ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
109572025-11-13T08:05:04.056ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce52979c-490f-4832-96fd-5f6a668ef480", "content-length": "734", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109582025-11-13T08:05:04.056ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
109592025-11-13T08:05:04.057ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3d42685-600a-4a98-be29-aa2c89a9a813", "content-length": "111", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109602025-11-13T08:05:04.057ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
109612025-11-13T08:05:04.057ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccfec966-ca6b-4922-8883-7b2bf62ab0ae", "content-length": "111", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109622025-11-13T08:05:04.057ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
109632025-11-13T08:05:04.057ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa274070-fb17-4fce-a66b-1a17bcfc7b72", "content-length": "163", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109642025-11-13T08:05:04.058ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
109652025-11-13T08:05:04.058ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1c8fec4-cfe1-4cf8-bedf-f195d949dc5f", "content-length": "163", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109662025-11-13T08:05:04.058ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
109672025-11-13T08:05:04.059ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "296320a0-d42c-47c2-ab15-7d489ac67315", "content-length": "166", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109682025-11-13T08:05:04.059ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
109692025-11-13T08:05:04.059ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e1f4347-37c8-4af9-8f71-c662eeda5458", "content-length": "166", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109702025-11-13T08:05:04.059ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
109712025-11-13T08:05:04.059ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "261c67da-67bb-4d04-a169-41bfba87ad74", "content-length": "702", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109722025-11-13T08:05:04.059ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
109732025-11-13T08:05:04.059ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6059e2c0-7fb4-41d4-8e87-53f5d49a7027", "content-length": "727", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109742025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
109752025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e62533b4-49e2-40f4-939b-219d17a7d842", "content-length": "729", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109762025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
109772025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77b7069a-9aa6-4e09-9555-fac2e739d877", "content-length": "728", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109782025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
109792025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38d887a1-f318-499b-88cf-6e6444405b18", "content-length": "722", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109802025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
109812025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "b34e7448-5c1d-4b4e-a425-402ee32adf6f", "content-length": "10", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109822025-11-13T08:05:04.060ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
109832025-11-13T08:05:04.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9a21381-c30c-4819-819c-bf88ae8181ce", "content-length": "144", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109842025-11-13T08:05:04.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
109852025-11-13T08:05:04.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abb039a1-61c6-4bcf-9aa2-b29c82a9d633", "content-length": "147", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109862025-11-13T08:05:04.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
109872025-11-13T08:05:04.061ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3f7bbf7-5963-4182-8bb0-da875570a8b9", "content-length": "109", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109882025-11-13T08:05:04.061ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
109892025-11-13T08:05:04.062ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "532b2417-f579-4a50-afa3-e82560930094", "content-length": "109", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109902025-11-13T08:05:04.062ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
109912025-11-13T08:05:04.062ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2eb7c9b0-6908-4af0-b5ce-4c36c7af74ac", "content-length": "163", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109922025-11-13T08:05:04.062ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
109932025-11-13T08:05:04.062ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9063953c-d4a7-4e76-92da-e7ac4dea3b80", "content-length": "163", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109942025-11-13T08:05:04.062ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
109952025-11-13T08:05:04.063ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21b59d00-720b-46f6-9793-4f33ea619f35", "content-length": "166", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109962025-11-13T08:05:04.063ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
109972025-11-13T08:05:04.063ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4335c9c-f505-4b00-812d-ae1c21e9ca3c", "content-length": "166", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
109982025-11-13T08:05:04.063ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
109992025-11-13T08:05:04.063ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d6cc6df-afa1-49ea-ad4a-5230af6be4d1", "content-length": "702", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110002025-11-13T08:05:04.063ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
110012025-11-13T08:05:04.064ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de7e7d7a-3278-41e4-b1fa-b995acc33ede", "content-length": "727", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110022025-11-13T08:05:04.064ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
110032025-11-13T08:05:04.064ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64186c4c-dafa-4d4f-be74-8859fc178e68", "content-length": "729", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110042025-11-13T08:05:04.064ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
110052025-11-13T08:05:04.064ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7e97c10-3ec7-4423-8099-176823b725ba", "content-length": "728", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110062025-11-13T08:05:04.064ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110072025-11-13T08:05:04.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
110082025-11-13T08:05:04.066ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110092025-11-13T08:05:04.069ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
110102025-11-13T08:05:04.069ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110112025-11-13T08:05:04.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
110122025-11-13T08:05:04.071ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110132025-11-13T08:05:04.073ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
110142025-11-13T08:05:04.073ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110152025-11-13T08:05:04.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
110162025-11-13T08:05:04.112ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
110172025-11-13T08:05:04.112ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
110182025-11-13T08:05:04.246ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fd62464-2dd4-49ea-a197-4d09fa11da5f", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110192025-11-13T08:05:04.248ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
110202025-11-13T08:05:04.248ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
110212025-11-13T08:05:04.248ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
110222025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
110232025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
110242025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
110252025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
110262025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
110272025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f2ab9fa-6331-4902-8db1-03dc0b24a3c4", "content-length": "64", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110282025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6f5496a-1a7d-48ae-90fa-8667c9b9062e", "content-length": "64", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110292025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
110302025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
110312025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "119033aa-c4b9-4910-81c5-14ec6bf90d3d", "content-length": "64", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110322025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
110332025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a970cd94-705c-47ac-85b9-a69a353122e4", "content-length": "64", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110342025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "52b5c4ea-36a6-4d22-b4d7-a00df5a7b9c5", "content-length": "64", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110352025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
110362025-11-13T08:05:04.248ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
110372025-11-13T08:05:04.286ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b6d6589-e0e3-4132-bdfe-e99bb24dea78", "content-length": "392531", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110382025-11-13T08:05:04.286ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "b221930e-9aea-42d6-b949-70f33c8a2dce", "content-length": "394490", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110392025-11-13T08:05:04.287ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "318941ee-f48e-47b0-98fa-c412206f23d5", "content-length": "401668", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110402025-11-13T08:05:04.288ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "138f19c6-d307-49a7-87fc-ac1134d2339e", "content-length": "390390", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110412025-11-13T08:05:04.289ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1efb0ef2-ee99-4ea6-805d-2a7a87d2e4b4", "content-length": "420007", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110422025-11-13T08:05:04.292ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
110432025-11-13T08:05:04.292ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
110442025-11-13T08:05:04.292ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
110452025-11-13T08:05:04.292ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
110462025-11-13T08:05:04.292ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
110472025-11-13T08:05:04.322ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
110482025-11-13T08:05:04.322ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
110492025-11-13T08:05:04.325ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd781ccd-cb6e-4824-aa79-b82411d6698a", "content-length": "122", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110502025-11-13T08:05:04.327ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
110512025-11-13T08:05:04.419ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
110522025-11-13T08:05:04.419ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
110532025-11-13T08:05:04.420ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b9c6577-dc47-47df-aebc-c3079717701c", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110542025-11-13T08:05:04.420ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
110552025-11-13T08:05:04.422ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
110562025-11-13T08:05:04.422ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
110572025-11-13T08:05:04.422ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "44c32273-423b-429f-a304-3889c6cd7e3d", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110582025-11-13T08:05:04.423ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
110592025-11-13T08:05:04.424ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
110602025-11-13T08:05:04.424ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
110612025-11-13T08:05:04.424ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc997968-366b-47a0-be2b-b9d261fdf547", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:05:04 GMT"} })
110622025-11-13T08:05:04.424ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
110632025-11-13T08:05:04.426ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
110642025-11-13T08:05:04.426ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
110652025-11-13T08:05:04.428ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110662025-11-13T08:05:04.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
110672025-11-13T08:05:04.430ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110682025-11-13T08:05:04.433ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
110692025-11-13T08:05:04.433ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110702025-11-13T08:05:04.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
110712025-11-13T08:05:04.628ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 54ba8229-42c5-4cf8-a019-8fd3109d6083 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
110722025-11-13T08:05:04.628ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 54ba8229-42c5-4cf8-a019-8fd3109d6083 time_started = 2025-11-13 08:05:04.055535 UTC
110732025-11-13T08:05:04.628ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 33 reason = Dependency
110742025-11-13T08:05:04.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
110752025-11-13T08:05:04.628ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 993.43884ms iteration = 8 status = {"collection_id":"54ba8229-42c5-4cf8-a019-8fd3109d6083","time_done":"2025-11-13 08:05:04.426468 UTC","time_started":"2025-11-13 08:05:04.055535 UTC"}
110762025-11-13T08:05:04.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
110772025-11-13T08:05:04.630ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110782025-11-13T08:05:04.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
110792025-11-13T08:05:04.633ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
110802025-11-13T08:05:04.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
110812025-11-13T08:05:04.636ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
110822025-11-13T08:05:04.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
110832025-11-13T08:05:04.694ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 66.11782ms iteration = 33 status = {"Loaded":{"collection_id":"c3f9dc97-7cb9-4750-a624-009dc8a84400","time_loaded":"2025-11-13T08:05:04.628462154Z","time_started":"2025-11-13T08:05:04.057911Z"}}
110842025-11-13T08:05:04.694ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 11 reason = Dependency
110852025-11-13T08:05:04.694ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 18 reason = Dependency
110862025-11-13T08:05:04.694ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 22 reason = Dependency
110872025-11-13T08:05:04.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
110882025-11-13T08:05:04.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110892025-11-13T08:05:04.694ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
110902025-11-13T08:05:04.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
110912025-11-13T08:05:04.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
110922025-11-13T08:05:04.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
110932025-11-13T08:05:04.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110942025-11-13T08:05:04.696ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110952025-11-13T08:05:04.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110962025-11-13T08:05:04.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
110972025-11-13T08:05:04.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
110982025-11-13T08:05:04.699ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
110992025-11-13T08:05:04.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111002025-11-13T08:05:04.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111012025-11-13T08:05:04.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111022025-11-13T08:05:04.700ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
111032025-11-13T08:05:04.700ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111042025-11-13T08:05:04.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111052025-11-13T08:05:04.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111062025-11-13T08:05:04.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111072025-11-13T08:05:04.701ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111082025-11-13T08:05:04.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111092025-11-13T08:05:04.703ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111102025-11-13T08:05:04.704ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111112025-11-13T08:05:04.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111122025-11-13T08:05:04.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111132025-11-13T08:05:04.705ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
111142025-11-13T08:05:04.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111152025-11-13T08:05:04.705ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111162025-11-13T08:05:04.706ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
111172025-11-13T08:05:04.706ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
111182025-11-13T08:05:04.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
111192025-11-13T08:05:04.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111202025-11-13T08:05:04.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111212025-11-13T08:05:04.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111222025-11-13T08:05:04.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111232025-11-13T08:05:04.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111242025-11-13T08:05:04.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111252025-11-13T08:05:04.711ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111262025-11-13T08:05:04.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111272025-11-13T08:05:04.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111282025-11-13T08:05:04.712ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
111292025-11-13T08:05:04.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111302025-11-13T08:05:04.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111312025-11-13T08:05:04.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111322025-11-13T08:05:04.718ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 23.464321ms iteration = 22 status = {"physical_disks_added":0}
111332025-11-13T08:05:04.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111342025-11-13T08:05:04.718ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111352025-11-13T08:05:04.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111362025-11-13T08:05:04.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111372025-11-13T08:05:04.721ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
111382025-11-13T08:05:04.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111392025-11-13T08:05:04.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111402025-11-13T08:05:04.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111412025-11-13T08:05:04.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111422025-11-13T08:05:04.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111432025-11-13T08:05:04.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111442025-11-13T08:05:04.732ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
111452025-11-13T08:05:04.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111462025-11-13T08:05:04.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111472025-11-13T08:05:04.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111482025-11-13T08:05:04.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111492025-11-13T08:05:04.736ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111502025-11-13T08:05:04.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111512025-11-13T08:05:04.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111522025-11-13T08:05:04.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111532025-11-13T08:05:04.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111542025-11-13T08:05:04.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111552025-11-13T08:05:04.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111562025-11-13T08:05:04.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111572025-11-13T08:05:04.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111582025-11-13T08:05:04.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111592025-11-13T08:05:04.741ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
111602025-11-13T08:05:04.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
111612025-11-13T08:05:04.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
111622025-11-13T08:05:04.743ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
111632025-11-13T08:05:04.743ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 48.82441ms iteration = 11 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"c3f9dc97-7cb9-4750-a624-009dc8a84400","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
111642025-11-13T08:05:04.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
111652025-11-13T08:05:04.743ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111662025-11-13T08:05:04.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111672025-11-13T08:05:04.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111682025-11-13T08:05:04.748ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111692025-11-13T08:05:04.748ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111702025-11-13T08:05:04.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111712025-11-13T08:05:04.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111722025-11-13T08:05:04.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111732025-11-13T08:05:04.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111742025-11-13T08:05:04.757ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
111752025-11-13T08:05:04.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111762025-11-13T08:05:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111772025-11-13T08:05:04.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111782025-11-13T08:05:04.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111792025-11-13T08:05:04.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111802025-11-13T08:05:04.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
111812025-11-13T08:05:04.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111822025-11-13T08:05:04.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111832025-11-13T08:05:04.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111842025-11-13T08:05:04.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111852025-11-13T08:05:04.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111862025-11-13T08:05:04.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
111872025-11-13T08:05:04.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111882025-11-13T08:05:04.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111892025-11-13T08:05:04.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111902025-11-13T08:05:04.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111912025-11-13T08:05:04.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111922025-11-13T08:05:04.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111932025-11-13T08:05:04.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111942025-11-13T08:05:04.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111952025-11-13T08:05:04.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
111962025-11-13T08:05:04.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
111972025-11-13T08:05:04.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111982025-11-13T08:05:04.797ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
111992025-11-13T08:05:04.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112002025-11-13T08:05:04.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112012025-11-13T08:05:04.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112022025-11-13T08:05:04.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112032025-11-13T08:05:04.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112042025-11-13T08:05:04.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112052025-11-13T08:05:04.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112062025-11-13T08:05:04.806ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
112072025-11-13T08:05:04.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112082025-11-13T08:05:04.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112092025-11-13T08:05:04.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112102025-11-13T08:05:04.810ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
112112025-11-13T08:05:04.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112122025-11-13T08:05:04.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112132025-11-13T08:05:04.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112142025-11-13T08:05:04.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112152025-11-13T08:05:04.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112162025-11-13T08:05:04.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112172025-11-13T08:05:04.814ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112182025-11-13T08:05:04.818ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
112192025-11-13T08:05:04.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112202025-11-13T08:05:04.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112212025-11-13T08:05:04.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112222025-11-13T08:05:04.820ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112232025-11-13T08:05:04.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112242025-11-13T08:05:04.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
112252025-11-13T08:05:04.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112262025-11-13T08:05:04.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112272025-11-13T08:05:04.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112282025-11-13T08:05:04.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112292025-11-13T08:05:04.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112302025-11-13T08:05:04.832ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
112312025-11-13T08:05:04.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112322025-11-13T08:05:04.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112332025-11-13T08:05:04.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112342025-11-13T08:05:04.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112352025-11-13T08:05:04.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112362025-11-13T08:05:04.838ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
112372025-11-13T08:05:04.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112382025-11-13T08:05:04.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112392025-11-13T08:05:04.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112402025-11-13T08:05:04.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112412025-11-13T08:05:04.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112422025-11-13T08:05:04.847ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
112432025-11-13T08:05:04.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112442025-11-13T08:05:04.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112452025-11-13T08:05:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112462025-11-13T08:05:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112472025-11-13T08:05:04.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112482025-11-13T08:05:04.853ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
112492025-11-13T08:05:04.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112502025-11-13T08:05:04.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112512025-11-13T08:05:04.854ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
112522025-11-13T08:05:04.854ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
112532025-11-13T08:05:04.855ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = dba99753-cae2-4e2d-9770-b10d7c6a3cb1 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
112542025-11-13T08:05:04.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112552025-11-13T08:05:04.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112562025-11-13T08:05:04.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112572025-11-13T08:05:04.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
112582025-11-13T08:05:04.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
112592025-11-13T08:05:04.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
112602025-11-13T08:05:04.863ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
112612025-11-13T08:05:04.863ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
112622025-11-13T08:05:04.863ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 168.493263ms iteration = 18 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
112632025-11-13T08:05:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 319 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 130dc50d-97d6-44d8-a7cf-000ef5d91455 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
112642025-11-13T08:05:07.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 74 reason = Timeout
112652025-11-13T08:05:07.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112662025-11-13T08:05:07.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112672025-11-13T08:05:07.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112682025-11-13T08:05:07.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
112692025-11-13T08:05:07.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112702025-11-13T08:05:07.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112712025-11-13T08:05:07.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
112722025-11-13T08:05:07.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.374355ms iteration = 74 status = {"config_updated":false}
112732025-11-13T08:05:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 38 reason = Timeout
112742025-11-13T08:05:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 75 reason = Timeout
112752025-11-13T08:05:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112762025-11-13T08:05:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112772025-11-13T08:05:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112782025-11-13T08:05:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112792025-11-13T08:05:12.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112802025-11-13T08:05:12.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112812025-11-13T08:05:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
112822025-11-13T08:05:12.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
112832025-11-13T08:05:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
112842025-11-13T08:05:12.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112852025-11-13T08:05:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
112862025-11-13T08:05:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112872025-11-13T08:05:12.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
112882025-11-13T08:05:12.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.547039ms iteration = 75 status = {"config_updated":false}
112892025-11-13T08:05:12.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112902025-11-13T08:05:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112912025-11-13T08:05:12.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112922025-11-13T08:05:12.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
112932025-11-13T08:05:12.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
112942025-11-13T08:05:12.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
112952025-11-13T08:05:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 38 reason = Timeout
112962025-11-13T08:05:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112972025-11-13T08:05:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
112982025-11-13T08:05:12.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
112992025-11-13T08:05:12.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113002025-11-13T08:05:12.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.961358ms iteration = 38 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
113012025-11-13T08:05:12.650ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
113022025-11-13T08:05:12.650ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.241176ms iteration = 38 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
113032025-11-13T08:05:15.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 9a903c93-cb0d-492e-a633-e472320f59e9 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
113042025-11-13T08:05:15.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 9a903c93-cb0d-492e-a633-e472320f59e9 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
113052025-11-13T08:05:15.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 9a903c93-cb0d-492e-a633-e472320f59e9 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
113062025-11-13T08:05:15.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3922 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 9a903c93-cb0d-492e-a633-e472320f59e9 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
113072025-11-13T08:05:15.477ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 5c337e80-42aa-46a8-b55e-c2b72377ef88 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
113082025-11-13T08:05:15.477ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 5c337e80-42aa-46a8-b55e-c2b72377ef88 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
113092025-11-13T08:05:15.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 5c337e80-42aa-46a8-b55e-c2b72377ef88 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
113102025-11-13T08:05:15.480ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2504 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 5c337e80-42aa-46a8-b55e-c2b72377ef88 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
113112025-11-13T08:05:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 259 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 7be2ff7e-2f97-4a0e-ae8f-2c5a447510b7 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
113122025-11-13T08:05:17.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 34 reason = Timeout
113132025-11-13T08:05:17.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 76 reason = Timeout
113142025-11-13T08:05:17.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113152025-11-13T08:05:17.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
113162025-11-13T08:05:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
113172025-11-13T08:05:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113182025-11-13T08:05:17.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113192025-11-13T08:05:17.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113202025-11-13T08:05:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
113212025-11-13T08:05:17.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113222025-11-13T08:05:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
113232025-11-13T08:05:17.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
113242025-11-13T08:05:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113252025-11-13T08:05:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
113262025-11-13T08:05:17.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
113272025-11-13T08:05:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.779803ms iteration = 76 status = {"config_updated":false}
113282025-11-13T08:05:17.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = c3f9dc97-7cb9-4750-a624-009dc8a84400 original_time_started = 2025-11-13 08:05:04.057911 UTC
113292025-11-13T08:05:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.252733ms iteration = 34 status = {"Loaded":{"collection_id":"c3f9dc97-7cb9-4750-a624-009dc8a84400","time_loaded":"2025-11-13T08:05:17.626523706Z","time_started":"2025-11-13T08:05:04.057911Z"}}
113302025-11-13T08:05:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 26 reason = Timeout
113312025-11-13T08:05:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 26 reason = Timeout
113322025-11-13T08:05:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
113332025-11-13T08:05:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113342025-11-13T08:05:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113352025-11-13T08:05:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
113362025-11-13T08:05:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113372025-11-13T08:05:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113382025-11-13T08:05:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113392025-11-13T08:05:17.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113402025-11-13T08:05:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
113412025-11-13T08:05:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
113422025-11-13T08:05:17.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
113432025-11-13T08:05:17.653ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
113442025-11-13T08:05:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.962768ms iteration = 26 status = "NoSitrep"
113452025-11-13T08:05:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113462025-11-13T08:05:17.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113472025-11-13T08:05:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
113482025-11-13T08:05:17.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
113492025-11-13T08:05:17.658ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113502025-11-13T08:05:17.659ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
113512025-11-13T08:05:17.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
113522025-11-13T08:05:17.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.360223ms iteration = 26 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
113532025-11-13T08:05:22.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 39 reason = Timeout
113542025-11-13T08:05:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 77 reason = Timeout
113552025-11-13T08:05:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113562025-11-13T08:05:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113572025-11-13T08:05:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113582025-11-13T08:05:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113592025-11-13T08:05:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113602025-11-13T08:05:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113612025-11-13T08:05:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
113622025-11-13T08:05:22.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113632025-11-13T08:05:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
113642025-11-13T08:05:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113652025-11-13T08:05:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113662025-11-13T08:05:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113672025-11-13T08:05:22.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
113682025-11-13T08:05:22.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.541938ms iteration = 77 status = {"config_updated":false}
113692025-11-13T08:05:22.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113702025-11-13T08:05:22.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113712025-11-13T08:05:22.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113722025-11-13T08:05:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
113732025-11-13T08:05:22.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113742025-11-13T08:05:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
113752025-11-13T08:05:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 39 reason = Timeout
113762025-11-13T08:05:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113772025-11-13T08:05:22.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113782025-11-13T08:05:22.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113792025-11-13T08:05:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113802025-11-13T08:05:22.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.05882ms iteration = 39 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
113812025-11-13T08:05:22.652ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
113822025-11-13T08:05:22.652ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.876648ms iteration = 39 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
113832025-11-13T08:05:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 312 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = f19ca5f3-dcc0-448e-ace1-d3f97669d5e2 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
113842025-11-13T08:05:27.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 78 reason = Timeout
113852025-11-13T08:05:27.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113862025-11-13T08:05:27.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113872025-11-13T08:05:27.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113882025-11-13T08:05:27.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
113892025-11-13T08:05:27.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
113902025-11-13T08:05:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
113912025-11-13T08:05:27.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
113922025-11-13T08:05:27.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.62925ms iteration = 78 status = {"config_updated":false}
113932025-11-13T08:05:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 14 reason = Timeout
113942025-11-13T08:05:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 14 reason = Timeout
113952025-11-13T08:05:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
113962025-11-13T08:05:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 14 reason = Timeout
113972025-11-13T08:05:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
113982025-11-13T08:05:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
113992025-11-13T08:05:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
114002025-11-13T08:05:32.610ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6340, tv_nsec: 963019557 } })
114012025-11-13T08:05:32.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6340, tv_nsec: 963180861 } })
114022025-11-13T08:05:32.611ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
114032025-11-13T08:05:32.611ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.965548ms iteration = 14 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
114042025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 21 reason = Timeout
114052025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 35 reason = Timeout
114062025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114072025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 40 reason = Timeout
114082025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114092025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
114102025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 79 reason = Timeout
114112025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 23 reason = Timeout
114122025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
114132025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114142025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114152025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
114162025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114172025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
114182025-11-13T08:05:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
114192025-11-13T08:05:32.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114202025-11-13T08:05:32.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114212025-11-13T08:05:32.631ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114222025-11-13T08:05:32.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114232025-11-13T08:05:32.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114242025-11-13T08:05:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
114252025-11-13T08:05:32.635ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114262025-11-13T08:05:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114272025-11-13T08:05:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
114282025-11-13T08:05:32.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114292025-11-13T08:05:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114302025-11-13T08:05:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
114312025-11-13T08:05:32.636ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
114322025-11-13T08:05:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
114332025-11-13T08:05:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
114342025-11-13T08:05:32.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114352025-11-13T08:05:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
114362025-11-13T08:05:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
114372025-11-13T08:05:32.638ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
114382025-11-13T08:05:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
114392025-11-13T08:05:32.638ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = c3f9dc97-7cb9-4750-a624-009dc8a84400 original_time_started = 2025-11-13 08:05:04.057911 UTC
114402025-11-13T08:05:32.638ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.847513ms iteration = 35 status = {"Loaded":{"collection_id":"c3f9dc97-7cb9-4750-a624-009dc8a84400","time_loaded":"2025-11-13T08:05:32.626102237Z","time_started":"2025-11-13T08:05:04.057911Z"}}
114412025-11-13T08:05:32.640ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114422025-11-13T08:05:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114432025-11-13T08:05:32.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114442025-11-13T08:05:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
114452025-11-13T08:05:32.641ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 14 reason = Timeout
114462025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 14 reason = Timeout
114472025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
114482025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
114492025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 14 reason = Timeout
114502025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 21 reason = Timeout
114512025-11-13T08:05:32.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
114522025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 40 reason = Timeout
114532025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114542025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
114552025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 14 reason = Timeout
114562025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
114572025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
114582025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 14 reason = Timeout
114592025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
114602025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
114612025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 14 reason = Timeout
114622025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114632025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
114642025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
114652025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
114662025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114672025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114682025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
114692025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
114702025-11-13T08:05:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
114712025-11-13T08:05:32.646ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
114722025-11-13T08:05:32.646ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114732025-11-13T08:05:32.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114742025-11-13T08:05:32.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
114752025-11-13T08:05:32.647ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114762025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114772025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 14 reason = Timeout
114782025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
114792025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 14 reason = Timeout
114802025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
114812025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 14 reason = Timeout
114822025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
114832025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 14 reason = Timeout
114842025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
114852025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 27 reason = Timeout
114862025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
114872025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 14 reason = Timeout
114882025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 14 reason = Timeout
114892025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
114902025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 27 reason = Timeout
114912025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
114922025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
114932025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114942025-11-13T08:05:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
114952025-11-13T08:05:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
114962025-11-13T08:05:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
114972025-11-13T08:05:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
114982025-11-13T08:05:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
114992025-11-13T08:05:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
115002025-11-13T08:05:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6341, tv_nsec: 1815310 } })
115012025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
115022025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115032025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
115042025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115052025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
115062025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115072025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115082025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115092025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115102025-11-13T08:05:32.650ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115112025-11-13T08:05:32.651ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.54196ms iteration = 14 status = {"errors":[],"finish_invoked_ok":[]}
115122025-11-13T08:05:32.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
115132025-11-13T08:05:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
115142025-11-13T08:05:32.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115152025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
115162025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115172025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
115182025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115192025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115202025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
115212025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115222025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
115232025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 29.362378ms iteration = 79 status = {"config_updated":false}
115242025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
115252025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
115262025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115272025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
115282025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115292025-11-13T08:05:32.655ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
115302025-11-13T08:05:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
115312025-11-13T08:05:32.656ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115322025-11-13T08:05:32.657ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
115332025-11-13T08:05:32.657ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
115342025-11-13T08:05:32.657ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
115352025-11-13T08:05:32.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
115362025-11-13T08:05:32.658ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115372025-11-13T08:05:32.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
115382025-11-13T08:05:32.659ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
115392025-11-13T08:05:32.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
115402025-11-13T08:05:32.659ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
115412025-11-13T08:05:32.659ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
115422025-11-13T08:05:32.659ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
115432025-11-13T08:05:32.660ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 33.942237ms iteration = 21 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
115442025-11-13T08:05:32.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
115452025-11-13T08:05:32.660ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115462025-11-13T08:05:32.660ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
115472025-11-13T08:05:32.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
115482025-11-13T08:05:32.661ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
115492025-11-13T08:05:32.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
115502025-11-13T08:05:32.662ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6341, tv_nsec: 14979599 } })
115512025-11-13T08:05:32.663ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
115522025-11-13T08:05:32.663ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115532025-11-13T08:05:32.663ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
115542025-11-13T08:05:32.663ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
115552025-11-13T08:05:32.663ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
115562025-11-13T08:05:32.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115572025-11-13T08:05:32.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
115582025-11-13T08:05:32.664ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
115592025-11-13T08:05:32.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
115602025-11-13T08:05:32.664ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 22.549643ms iteration = 40 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
115612025-11-13T08:05:32.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
115622025-11-13T08:05:32.665ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115632025-11-13T08:05:32.665ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
115642025-11-13T08:05:32.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115652025-11-13T08:05:32.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
115662025-11-13T08:05:32.669ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 21.064154ms iteration = 27 status = "NoSitrep"
115672025-11-13T08:05:32.670ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115682025-11-13T08:05:32.672ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 30.697884ms iteration = 14 status = 0
115692025-11-13T08:05:32.673ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
115702025-11-13T08:05:32.674ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
115712025-11-13T08:05:32.674ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 25.703006ms iteration = 27 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
115722025-11-13T08:05:32.675ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115732025-11-13T08:05:32.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115742025-11-13T08:05:32.677ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115752025-11-13T08:05:32.681ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
115762025-11-13T08:05:32.681ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115772025-11-13T08:05:32.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115782025-11-13T08:05:32.685ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115792025-11-13T08:05:32.689ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
115802025-11-13T08:05:32.689ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
115812025-11-13T08:05:32.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
115822025-11-13T08:05:32.690ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
115832025-11-13T08:05:32.690ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.426847ms iteration = 40 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
115842025-11-13T08:05:32.722ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 73.702719ms iteration = 14 status = {"errors":[],"garbage_collect_requested":[]}
115852025-11-13T08:05:32.722ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
115862025-11-13T08:05:32.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
115872025-11-13T08:05:32.724ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
115882025-11-13T08:05:32.724ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
115892025-11-13T08:05:32.725ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 76.725788ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
115902025-11-13T08:05:32.726ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 100.448495ms iteration = 23 status = {"physical_disks_added":0}
115912025-11-13T08:05:32.727ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 85.171985ms iteration = 14 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
115922025-11-13T08:05:32.732ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 83.363589ms iteration = 14 status = {"errors":[],"requests_created_ok":[]}
115932025-11-13T08:05:32.732ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 83.642095ms iteration = 14 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
115942025-11-13T08:05:32.741ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
115952025-11-13T08:05:32.741ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
115962025-11-13T08:05:32.741ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 99.094458ms iteration = 14 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
115972025-11-13T08:05:32.754ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
115982025-11-13T08:05:32.754ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 112.223607ms iteration = 14 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
115992025-11-13T08:05:32.776ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
116002025-11-13T08:05:32.777ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 170.89772ms iteration = 14 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
116012025-11-13T08:05:32.828ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
116022025-11-13T08:05:32.830ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8c7db18-e61a-49ff-8e6f-3db5df869f41", "content-length": "691", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116032025-11-13T08:05:32.830ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
116042025-11-13T08:05:32.830ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
116052025-11-13T08:05:32.830ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
116062025-11-13T08:05:32.832ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 190.679128ms iteration = 14 status = {}
116072025-11-13T08:05:32.839ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
116082025-11-13T08:05:32.839ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
116092025-11-13T08:05:32.840ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "12c20035-c00b-44bb-8981-0f6615fffc5a", "content-length": "372", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116102025-11-13T08:05:32.840ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116112025-11-13T08:05:32.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116122025-11-13T08:05:32.840ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
116132025-11-13T08:05:32.840ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
116142025-11-13T08:05:32.841ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd894f99-305c-4df9-9f96-577e38985424", "content-length": "26", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116152025-11-13T08:05:32.841ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
116162025-11-13T08:05:32.842ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
116172025-11-13T08:05:32.853ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
116182025-11-13T08:05:32.853ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
116192025-11-13T08:05:32.853ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9908ef8c-233d-405e-b293-3378185cc436", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116202025-11-13T08:05:32.853ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116212025-11-13T08:05:32.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116222025-11-13T08:05:32.856ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116232025-11-13T08:05:32.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116242025-11-13T08:05:32.859ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116252025-11-13T08:05:32.859ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
116262025-11-13T08:05:32.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116272025-11-13T08:05:32.860ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "012c9572-708e-4fea-9ef2-656aa30400c0", "content-length": "78", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116282025-11-13T08:05:32.860ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
116292025-11-13T08:05:32.860ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
116302025-11-13T08:05:32.860ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
116312025-11-13T08:05:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
116322025-11-13T08:05:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
116332025-11-13T08:05:32.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
116342025-11-13T08:05:32.861ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
116352025-11-13T08:05:32.861ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
116362025-11-13T08:05:32.862ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
116372025-11-13T08:05:32.863ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
116382025-11-13T08:05:32.863ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
116392025-11-13T08:05:32.863ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "a77172f2-8aa2-4f23-8cd4-7c7d1ad18be8", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116402025-11-13T08:05:32.864ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
116412025-11-13T08:05:32.864ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
116422025-11-13T08:05:32.864ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 222.729519ms iteration = 21 status = {}
116432025-11-13T08:05:32.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
116442025-11-13T08:05:32.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
116452025-11-13T08:05:32.865ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
116462025-11-13T08:05:32.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
116472025-11-13T08:05:32.866ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
116482025-11-13T08:05:32.866ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
116492025-11-13T08:05:32.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
116502025-11-13T08:05:32.867ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
116512025-11-13T08:05:32.867ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
116522025-11-13T08:05:32.867ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f547a816-1845-4f0e-ac51-3377ccb65774", "content-length": "81", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116532025-11-13T08:05:32.868ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
116542025-11-13T08:05:32.868ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
116552025-11-13T08:05:32.869ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd92e859-4a0f-409c-960b-5cf58b0fc8ba", "content-length": "81", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116562025-11-13T08:05:32.870ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
116572025-11-13T08:05:32.870ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
116582025-11-13T08:05:32.870ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec4dac17-817a-4b1a-aa21-f948eb5faf45", "content-length": "81", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116592025-11-13T08:05:32.872ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 223.770039ms iteration = 14 status = {"disabled":false,"errors":[],"sps":[]}
116602025-11-13T08:05:32.881ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
116612025-11-13T08:05:32.881ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
116622025-11-13T08:05:32.882ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f88068b-e58c-469f-ad68-cbf3bb18211e", "content-length": "26", "date": "Thu, 13 Nov 2025 08:05:32 GMT"} })
116632025-11-13T08:05:32.882ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
116642025-11-13T08:05:32.883ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
116652025-11-13T08:05:32.909ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 302.222641ms iteration = 14 status = {}
116662025-11-13T08:05:32.918ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6341, tv_nsec: 271288928 } })
116672025-11-13T08:05:32.986ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116682025-11-13T08:05:32.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116692025-11-13T08:05:32.988ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
116702025-11-13T08:05:32.988ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116712025-11-13T08:05:32.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116722025-11-13T08:05:33.002ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
116732025-11-13T08:05:33.002ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
116742025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
116752025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
116762025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
116772025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
116782025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
116792025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
116802025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
116812025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
116822025-11-13T08:05:33.003ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
116832025-11-13T08:05:33.003ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116842025-11-13T08:05:33.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116852025-11-13T08:05:33.006ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
116862025-11-13T08:05:33.006ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
116872025-11-13T08:05:33.204ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "200106e7-7f5a-430f-9a36-fdfb07aecdac", "date": "Thu, 13 Nov 2025 08:05:33 GMT"} })
116882025-11-13T08:05:33.204ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
116892025-11-13T08:05:33.204ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
116902025-11-13T08:05:33.204ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb0d0f4b-9035-4ac9-a19f-74aefe30b82d", "content-length": "682", "date": "Thu, 13 Nov 2025 08:05:33 GMT"} })
116912025-11-13T08:05:33.205ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116922025-11-13T08:05:33.205ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116932025-11-13T08:05:33.208ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116942025-11-13T08:05:33.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116952025-11-13T08:05:33.211ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116962025-11-13T08:05:33.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116972025-11-13T08:05:33.213ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
116982025-11-13T08:05:33.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
116992025-11-13T08:05:33.220ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 578.593685ms iteration = 14 status = {}
117002025-11-13T08:05:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 459 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = c4da1046-0b1d-43c0-8175-6d356cdf0da9 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
117012025-11-13T08:05:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 80 reason = Timeout
117022025-11-13T08:05:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117032025-11-13T08:05:37.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117042025-11-13T08:05:37.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117052025-11-13T08:05:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
117062025-11-13T08:05:37.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117072025-11-13T08:05:37.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117082025-11-13T08:05:37.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
117092025-11-13T08:05:37.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.211842ms iteration = 80 status = {"config_updated":false}
117102025-11-13T08:05:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 41 reason = Timeout
117112025-11-13T08:05:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 81 reason = Timeout
117122025-11-13T08:05:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117132025-11-13T08:05:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117142025-11-13T08:05:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117152025-11-13T08:05:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117162025-11-13T08:05:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117172025-11-13T08:05:42.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117182025-11-13T08:05:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
117192025-11-13T08:05:42.633ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117202025-11-13T08:05:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
117212025-11-13T08:05:42.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117222025-11-13T08:05:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117232025-11-13T08:05:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117242025-11-13T08:05:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117252025-11-13T08:05:42.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117262025-11-13T08:05:42.635ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
117272025-11-13T08:05:42.635ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.829363ms iteration = 81 status = {"config_updated":false}
117282025-11-13T08:05:42.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117292025-11-13T08:05:42.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
117302025-11-13T08:05:42.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117312025-11-13T08:05:42.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117322025-11-13T08:05:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 41 reason = Timeout
117332025-11-13T08:05:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117342025-11-13T08:05:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117352025-11-13T08:05:42.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
117362025-11-13T08:05:42.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
117372025-11-13T08:05:42.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.55525ms iteration = 41 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
117382025-11-13T08:05:42.653ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
117392025-11-13T08:05:42.653ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.761026ms iteration = 41 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
117402025-11-13T08:05:45.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 26189e58-4c0c-4770-b1d5-20a6cd8aac06 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
117412025-11-13T08:05:45.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 26189e58-4c0c-4770-b1d5-20a6cd8aac06 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
117422025-11-13T08:05:45.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 26189e58-4c0c-4770-b1d5-20a6cd8aac06 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
117432025-11-13T08:05:45.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 1956 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = 26189e58-4c0c-4770-b1d5-20a6cd8aac06 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
117442025-11-13T08:05:45.475ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = f0eedcbe-7fec-49b0-aeb0-ef97c11c0376 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
117452025-11-13T08:05:45.475ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = f0eedcbe-7fec-49b0-aeb0-ef97c11c0376 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
117462025-11-13T08:05:45.475ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = f0eedcbe-7fec-49b0-aeb0-ef97c11c0376 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
117472025-11-13T08:05:45.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2148 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:58555 req_id = f0eedcbe-7fec-49b0-aeb0-ef97c11c0376 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
117482025-11-13T08:05:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 276 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 7006b412-9659-4f6d-9ad7-5ee085d03217 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
117492025-11-13T08:05:47.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 36 reason = Timeout
117502025-11-13T08:05:47.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 82 reason = Timeout
117512025-11-13T08:05:47.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117522025-11-13T08:05:47.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
117532025-11-13T08:05:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
117542025-11-13T08:05:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117552025-11-13T08:05:47.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117562025-11-13T08:05:47.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117572025-11-13T08:05:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
117582025-11-13T08:05:47.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
117592025-11-13T08:05:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
117602025-11-13T08:05:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
117612025-11-13T08:05:47.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117622025-11-13T08:05:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117632025-11-13T08:05:47.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
117642025-11-13T08:05:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.284763ms iteration = 82 status = {"config_updated":false}
117652025-11-13T08:05:47.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = c3f9dc97-7cb9-4750-a624-009dc8a84400 original_time_started = 2025-11-13 08:05:04.057911 UTC
117662025-11-13T08:05:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.393885ms iteration = 36 status = {"Loaded":{"collection_id":"c3f9dc97-7cb9-4750-a624-009dc8a84400","time_loaded":"2025-11-13T08:05:47.626075264Z","time_started":"2025-11-13T08:05:04.057911Z"}}
117672025-11-13T08:05:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 28 reason = Timeout
117682025-11-13T08:05:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 28 reason = Timeout
117692025-11-13T08:05:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
117702025-11-13T08:05:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
117712025-11-13T08:05:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
117722025-11-13T08:05:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
117732025-11-13T08:05:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
117742025-11-13T08:05:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
117752025-11-13T08:05:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117762025-11-13T08:05:47.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117772025-11-13T08:05:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
117782025-11-13T08:05:47.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
117792025-11-13T08:05:47.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
117802025-11-13T08:05:47.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
117812025-11-13T08:05:47.655ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.366755ms iteration = 28 status = "NoSitrep"
117822025-11-13T08:05:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
117832025-11-13T08:05:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
117842025-11-13T08:05:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
117852025-11-13T08:05:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
117862025-11-13T08:05:47.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117872025-11-13T08:05:47.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
117882025-11-13T08:05:47.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
117892025-11-13T08:05:47.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.350324ms iteration = 28 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
117902025-11-13T08:05:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 42 reason = Timeout
117912025-11-13T08:05:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 83 reason = Timeout
117922025-11-13T08:05:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
117932025-11-13T08:05:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
117942025-11-13T08:05:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
117952025-11-13T08:05:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
117962025-11-13T08:05:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117972025-11-13T08:05:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117982025-11-13T08:05:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
117992025-11-13T08:05:52.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
118002025-11-13T08:05:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
118012025-11-13T08:05:52.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118022025-11-13T08:05:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
118032025-11-13T08:05:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
118042025-11-13T08:05:52.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118052025-11-13T08:05:52.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
118062025-11-13T08:05:52.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
118072025-11-13T08:05:52.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.583189ms iteration = 83 status = {"config_updated":false}
118082025-11-13T08:05:52.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118092025-11-13T08:05:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
118102025-11-13T08:05:52.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
118112025-11-13T08:05:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
118122025-11-13T08:05:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 42 reason = Timeout
118132025-11-13T08:05:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
118142025-11-13T08:05:52.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
118152025-11-13T08:05:52.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
118162025-11-13T08:05:52.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
118172025-11-13T08:05:52.646ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.593201ms iteration = 42 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
118182025-11-13T08:05:52.649ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
118192025-11-13T08:05:52.649ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.126875ms iteration = 42 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
118202025-11-13T08:05:55.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 324 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 07a82e31-8c05-4fd8-900a-07b1cdf86582 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
118212025-11-13T08:05:57.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 84 reason = Timeout
118222025-11-13T08:05:57.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
118232025-11-13T08:05:57.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
118242025-11-13T08:05:57.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118252025-11-13T08:05:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
118262025-11-13T08:05:57.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
118272025-11-13T08:05:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
118282025-11-13T08:05:57.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
118292025-11-13T08:05:57.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.10113ms iteration = 84 status = {"config_updated":false}
118302025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 8 reason = Timeout
118312025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 8 reason = Timeout
118322025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 15 reason = Timeout
118332025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 10 reason = Timeout
118342025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118352025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 10 reason = Timeout
118362025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 15 reason = Timeout
118372025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 15 reason = Timeout
118382025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
118392025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
118402025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 8 reason = Timeout
118412025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 8 reason = Timeout
118422025-11-13T08:06:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:56:02.607703081 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
118432025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
118442025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 8 reason = Timeout
118452025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
118462025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 8 reason = Timeout
118472025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118482025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
118492025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
118502025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 9 reason = Timeout
118512025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
118522025-11-13T08:06:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
118532025-11-13T08:06:02.607ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = ExternalNetworking step id = Ensure
118542025-11-13T08:06:02.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118552025-11-13T08:06:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6370, tv_nsec: 960571357 } })
118562025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
118572025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
118582025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118592025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118602025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118612025-11-13T08:06:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6370, tv_nsec: 960803482 } })
118622025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
118632025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118642025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
118652025-11-13T08:06:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.339336ms iteration = 8 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
118662025-11-13T08:06:02.609ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
118672025-11-13T08:06:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.510339ms iteration = 8 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
118682025-11-13T08:06:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118692025-11-13T08:06:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118702025-11-13T08:06:02.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
118712025-11-13T08:06:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
118722025-11-13T08:06:02.611ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:56:02.607703081 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
118732025-11-13T08:06:02.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118742025-11-13T08:06:02.611ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.249494ms iteration = 8 status = {"expiration":"2025-11-13T07:56:02.607703081Z","pruned":[]}
118752025-11-13T08:06:02.615ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118762025-11-13T08:06:02.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118772025-11-13T08:06:02.617ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
118782025-11-13T08:06:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 8 reason = Timeout
118792025-11-13T08:06:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
118802025-11-13T08:06:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 19 reason = Timeout
118812025-11-13T08:06:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
118822025-11-13T08:06:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
118832025-11-13T08:06:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
118842025-11-13T08:06:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
118852025-11-13T08:06:02.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
118862025-11-13T08:06:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
118872025-11-13T08:06:02.620ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
118882025-11-13T08:06:02.620ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
118892025-11-13T08:06:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
118902025-11-13T08:06:02.622ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
118912025-11-13T08:06:02.622ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
118922025-11-13T08:06:02.622ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
118932025-11-13T08:06:02.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
118942025-11-13T08:06:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
118952025-11-13T08:06:02.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
118962025-11-13T08:06:02.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118972025-11-13T08:06:02.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 22 reason = Timeout
118982025-11-13T08:06:02.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
118992025-11-13T08:06:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
119002025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
119012025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 19.411641ms iteration = 15 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
119022025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
119032025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
119042025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
119052025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 37 reason = Timeout
119062025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
119072025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
119082025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 24 reason = Timeout
119092025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
119102025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119112025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 43 reason = Timeout
119122025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 85 reason = Timeout
119132025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
119142025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119152025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
119162025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
119172025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
119182025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119192025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
119202025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
119212025-11-13T08:06:02.627ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6370, tv_nsec: 979977958 } })
119222025-11-13T08:06:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
119232025-11-13T08:06:02.628ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119242025-11-13T08:06:02.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119252025-11-13T08:06:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119262025-11-13T08:06:02.629ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6370, tv_nsec: 981374246 } })
119272025-11-13T08:06:02.629ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
119282025-11-13T08:06:02.629ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119292025-11-13T08:06:02.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119302025-11-13T08:06:02.630ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119312025-11-13T08:06:02.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119322025-11-13T08:06:02.631ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119332025-11-13T08:06:02.631ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119342025-11-13T08:06:02.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119352025-11-13T08:06:02.631ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119362025-11-13T08:06:02.632ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119372025-11-13T08:06:02.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119382025-11-13T08:06:02.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119392025-11-13T08:06:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
119402025-11-13T08:06:02.638ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
119412025-11-13T08:06:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
119422025-11-13T08:06:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
119432025-11-13T08:06:02.638ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
119442025-11-13T08:06:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
119452025-11-13T08:06:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
119462025-11-13T08:06:02.639ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
119472025-11-13T08:06:02.639ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
119482025-11-13T08:06:02.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
119492025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
119502025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
119512025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
119522025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
119532025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
119542025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
119552025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
119562025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
119572025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
119582025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
119592025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
119602025-11-13T08:06:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
119612025-11-13T08:06:02.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
119622025-11-13T08:06:02.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
119632025-11-13T08:06:02.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
119642025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
119652025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
119662025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
119672025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 8 reason = Timeout
119682025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
119692025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 15 reason = Timeout
119702025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 22 reason = Timeout
119712025-11-13T08:06:02.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
119722025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 15 reason = Timeout
119732025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
119742025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
119752025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
119762025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 15 reason = Timeout
119772025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
119782025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 43 reason = Timeout
119792025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
119802025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 15 reason = Timeout
119812025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
119822025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
119832025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 15 reason = Timeout
119842025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
119852025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 15 reason = Timeout
119862025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
119872025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 8 reason = Timeout
119882025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
119892025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
119902025-11-13T08:06:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
119912025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
119922025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
119932025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
119942025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
119952025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
119962025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
119972025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
119982025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
119992025-11-13T08:06:02.643ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
120002025-11-13T08:06:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
120012025-11-13T08:06:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
120022025-11-13T08:06:02.644ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 8 reason = Timeout
120032025-11-13T08:06:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
120042025-11-13T08:06:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
120052025-11-13T08:06:02.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120062025-11-13T08:06:02.646ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
120072025-11-13T08:06:02.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
120082025-11-13T08:06:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120092025-11-13T08:06:02.647ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
120102025-11-13T08:06:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
120112025-11-13T08:06:02.647ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120122025-11-13T08:06:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
120132025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 8 reason = Timeout
120142025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
120152025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 15 reason = Timeout
120162025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 15 reason = Timeout
120172025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
120182025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
120192025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = c3f9dc97-7cb9-4750-a624-009dc8a84400 original_time_started = 2025-11-13 08:05:04.057911 UTC
120202025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 29 reason = Timeout
120212025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 21.795279ms iteration = 37 status = {"Loaded":{"collection_id":"c3f9dc97-7cb9-4750-a624-009dc8a84400","time_loaded":"2025-11-13T08:06:02.627031212Z","time_started":"2025-11-13T08:05:04.057911Z"}}
120222025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
120232025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 29 reason = Timeout
120242025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 15 reason = Timeout
120252025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
120262025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 15 reason = Timeout
120272025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
120282025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 15 reason = Timeout
120292025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
120302025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 8 reason = Timeout
120312025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
120322025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
120332025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 15 reason = Timeout
120342025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
120352025-11-13T08:06:02.648ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 8 reason = Timeout
120362025-11-13T08:06:02.648ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
120372025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
120382025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
120392025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
120402025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
120412025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
120422025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
120432025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
120442025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
120452025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
120462025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
120472025-11-13T08:06:02.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6371, tv_nsec: 1805568 } })
120482025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 4.650582ms iteration = 8 status = {"errors":[],"found_port_ok":[]}
120492025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
120502025-11-13T08:06:02.649ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 22.579054ms iteration = 85 status = {"config_updated":false}
120512025-11-13T08:06:02.651ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
120522025-11-13T08:06:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
120532025-11-13T08:06:02.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
120542025-11-13T08:06:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
120552025-11-13T08:06:02.663ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
120562025-11-13T08:06:02.663ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 14.795631ms iteration = 8 status = {"by_rx":{},"error":null}
120572025-11-13T08:06:02.663ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
120582025-11-13T08:06:02.665ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120592025-11-13T08:06:02.670ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
120602025-11-13T08:06:02.670ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
120612025-11-13T08:06:02.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
120622025-11-13T08:06:02.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120632025-11-13T08:06:02.672ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
120642025-11-13T08:06:02.675ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120652025-11-13T08:06:02.677ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
120662025-11-13T08:06:02.677ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
120672025-11-13T08:06:02.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
120682025-11-13T08:06:02.679ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
120692025-11-13T08:06:02.679ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
120702025-11-13T08:06:02.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
120712025-11-13T08:06:02.696ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
120722025-11-13T08:06:02.701ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 53.041743ms iteration = 15 status = {"errors":[],"garbage_collect_requested":[]}
120732025-11-13T08:06:02.705ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 56.47712ms iteration = 15 status = {"errors":[],"finish_invoked_ok":[]}
120742025-11-13T08:06:02.708ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120752025-11-13T08:06:02.709ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
120762025-11-13T08:06:02.709ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
120772025-11-13T08:06:02.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
120782025-11-13T08:06:02.713ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
120792025-11-13T08:06:02.713ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
120802025-11-13T08:06:02.713ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
120812025-11-13T08:06:02.713ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
120822025-11-13T08:06:02.717ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
120832025-11-13T08:06:02.733ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
120842025-11-13T08:06:02.738ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
120852025-11-13T08:06:02.739ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
120862025-11-13T08:06:02.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
120872025-11-13T08:06:02.741ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
120882025-11-13T08:06:02.741ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
120892025-11-13T08:06:02.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
120902025-11-13T08:06:02.742ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
120912025-11-13T08:06:02.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
120922025-11-13T08:06:02.743ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120932025-11-13T08:06:02.745ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
120942025-11-13T08:06:02.745ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120952025-11-13T08:06:02.746ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = SledList step id = Fetch
120962025-11-13T08:06:02.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
120972025-11-13T08:06:02.746ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
120982025-11-13T08:06:02.748ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 105.231678ms iteration = 15 status = 0
120992025-11-13T08:06:02.748ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121002025-11-13T08:06:02.748ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
121012025-11-13T08:06:02.748ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
121022025-11-13T08:06:02.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
121032025-11-13T08:06:02.750ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121042025-11-13T08:06:02.750ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121052025-11-13T08:06:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121062025-11-13T08:06:02.751ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
121072025-11-13T08:06:02.751ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
121082025-11-13T08:06:02.751ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 108.808469ms iteration = 8 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
121092025-11-13T08:06:02.751ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6371, tv_nsec: 104245632 } })
121102025-11-13T08:06:02.752ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
121112025-11-13T08:06:02.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
121122025-11-13T08:06:02.752ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
121132025-11-13T08:06:02.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
121142025-11-13T08:06:02.753ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121152025-11-13T08:06:02.754ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 105.774099ms iteration = 29 status = "NoSitrep"
121162025-11-13T08:06:02.756ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 114.060223ms iteration = 43 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
121172025-11-13T08:06:02.756ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121182025-11-13T08:06:02.757ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
121192025-11-13T08:06:02.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
121202025-11-13T08:06:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
121212025-11-13T08:06:02.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
121222025-11-13T08:06:02.761ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
121232025-11-13T08:06:02.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
121242025-11-13T08:06:02.762ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
121252025-11-13T08:06:02.762ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
121262025-11-13T08:06:02.762ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
121272025-11-13T08:06:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
121282025-11-13T08:06:02.773ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121292025-11-13T08:06:02.776ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
121302025-11-13T08:06:02.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
121312025-11-13T08:06:02.779ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
121322025-11-13T08:06:02.779ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
121332025-11-13T08:06:02.779ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
121342025-11-13T08:06:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
121352025-11-13T08:06:02.779ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
121362025-11-13T08:06:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
121372025-11-13T08:06:02.779ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
121382025-11-13T08:06:02.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121392025-11-13T08:06:02.782ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121402025-11-13T08:06:02.783ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 134.622887ms iteration = 15 status = {"errors":[],"requests_created_ok":[]}
121412025-11-13T08:06:02.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121422025-11-13T08:06:02.785ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
121432025-11-13T08:06:02.785ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
121442025-11-13T08:06:02.785ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 166.807629ms iteration = 8 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
121452025-11-13T08:06:02.785ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121462025-11-13T08:06:02.786ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121472025-11-13T08:06:02.787ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
121482025-11-13T08:06:02.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
121492025-11-13T08:06:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
121502025-11-13T08:06:02.790ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
121512025-11-13T08:06:02.790ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 163.144717ms iteration = 43 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
121522025-11-13T08:06:02.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121532025-11-13T08:06:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
121542025-11-13T08:06:02.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
121552025-11-13T08:06:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
121562025-11-13T08:06:02.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
121572025-11-13T08:06:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
121582025-11-13T08:06:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
121592025-11-13T08:06:02.794ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
121602025-11-13T08:06:02.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
121612025-11-13T08:06:02.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
121622025-11-13T08:06:02.796ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
121632025-11-13T08:06:02.796ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
121642025-11-13T08:06:02.796ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
121652025-11-13T08:06:02.796ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 188.307052ms iteration = 8 status = {"generation":2}
121662025-11-13T08:06:02.804ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
121672025-11-13T08:06:02.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
121682025-11-13T08:06:02.810ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121692025-11-13T08:06:02.811ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = DeployNexusRecords step id = Ensure
121702025-11-13T08:06:02.811ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
121712025-11-13T08:06:02.812ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
121722025-11-13T08:06:02.812ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
121732025-11-13T08:06:02.812ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
121742025-11-13T08:06:02.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
121752025-11-13T08:06:02.812ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 204.342767ms iteration = 8 status = {"generation":1}
121762025-11-13T08:06:02.815ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121772025-11-13T08:06:02.815ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
121782025-11-13T08:06:02.815ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
121792025-11-13T08:06:02.815ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 166.80981ms iteration = 8 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
121802025-11-13T08:06:02.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
121812025-11-13T08:06:02.818ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
121822025-11-13T08:06:02.818ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
121832025-11-13T08:06:02.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
121842025-11-13T08:06:02.825ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
121852025-11-13T08:06:02.825ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
121862025-11-13T08:06:02.825ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
121872025-11-13T08:06:02.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
121882025-11-13T08:06:02.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
121892025-11-13T08:06:02.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
121902025-11-13T08:06:02.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121912025-11-13T08:06:02.834ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
121922025-11-13T08:06:02.834ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
121932025-11-13T08:06:02.836ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 228.88857ms iteration = 15 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
121942025-11-13T08:06:02.841ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
121952025-11-13T08:06:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
121962025-11-13T08:06:02.844ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 201.969371ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
121972025-11-13T08:06:02.854ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
121982025-11-13T08:06:02.857ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
121992025-11-13T08:06:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
122002025-11-13T08:06:02.863ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
122012025-11-13T08:06:02.863ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 255.890221ms iteration = 8 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
122022025-11-13T08:06:02.864ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
122032025-11-13T08:06:02.864ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
122042025-11-13T08:06:02.864ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
122052025-11-13T08:06:02.865ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7cf66c5-e578-4f29-ac79-33c041280083", "content-length": "26", "date": "Thu, 13 Nov 2025 08:06:02 GMT"} })
122062025-11-13T08:06:02.865ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
122072025-11-13T08:06:02.866ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
122082025-11-13T08:06:02.866ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 240.535878ms iteration = 22 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
122092025-11-13T08:06:02.867ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
122102025-11-13T08:06:02.867ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122112025-11-13T08:06:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
122122025-11-13T08:06:02.874ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
122132025-11-13T08:06:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
122142025-11-13T08:06:02.878ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
122152025-11-13T08:06:02.878ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 236.373717ms iteration = 15 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
122162025-11-13T08:06:02.882ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
122172025-11-13T08:06:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
122182025-11-13T08:06:02.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122192025-11-13T08:06:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122202025-11-13T08:06:02.884ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122212025-11-13T08:06:02.886ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
122222025-11-13T08:06:02.887ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
122232025-11-13T08:06:02.887ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 238.638551ms iteration = 29 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
122242025-11-13T08:06:02.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122252025-11-13T08:06:02.900ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 251.312851ms iteration = 15 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
122262025-11-13T08:06:02.901ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
122272025-11-13T08:06:02.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122282025-11-13T08:06:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122292025-11-13T08:06:02.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122302025-11-13T08:06:02.910ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
122312025-11-13T08:06:02.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122322025-11-13T08:06:02.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122332025-11-13T08:06:02.914ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = SledAgent step id = Ensure
122342025-11-13T08:06:02.916ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
122352025-11-13T08:06:02.916ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
122362025-11-13T08:06:02.916ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
122372025-11-13T08:06:02.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
122382025-11-13T08:06:02.920ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
122392025-11-13T08:06:02.920ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
122402025-11-13T08:06:02.920ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 277.543247ms iteration = 15 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
122412025-11-13T08:06:02.927ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
122422025-11-13T08:06:02.927ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "14c578a9-15b6-402e-ab1c-e0d661fadd36", "content-length": "78", "date": "Thu, 13 Nov 2025 08:06:02 GMT"} })
122432025-11-13T08:06:02.928ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
122442025-11-13T08:06:02.928ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
122452025-11-13T08:06:02.928ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
122462025-11-13T08:06:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
122472025-11-13T08:06:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
122482025-11-13T08:06:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
122492025-11-13T08:06:02.931ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
122502025-11-13T08:06:02.933ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
122512025-11-13T08:06:02.934ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
122522025-11-13T08:06:02.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
122532025-11-13T08:06:02.935ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
122542025-11-13T08:06:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
122552025-11-13T08:06:02.938ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
122562025-11-13T08:06:02.938ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
122572025-11-13T08:06:02.938ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
122582025-11-13T08:06:02.938ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
122592025-11-13T08:06:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
122602025-11-13T08:06:02.939ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
122612025-11-13T08:06:02.940ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 291.553182ms iteration = 15 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
122622025-11-13T08:06:02.941ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
122632025-11-13T08:06:02.941ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
122642025-11-13T08:06:02.942ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "62300b63-0eae-4e22-b28b-e5172acdbc54", "content-length": "81", "date": "Thu, 13 Nov 2025 08:06:02 GMT"} })
122652025-11-13T08:06:02.958ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
122662025-11-13T08:06:02.958ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
122672025-11-13T08:06:02.959ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ae6192e-321b-4405-ac0f-56887c9b5119", "content-length": "26", "date": "Thu, 13 Nov 2025 08:06:02 GMT"} })
122682025-11-13T08:06:02.959ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
122692025-11-13T08:06:02.959ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
122702025-11-13T08:06:02.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
122712025-11-13T08:06:02.961ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
122722025-11-13T08:06:02.962ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122732025-11-13T08:06:02.966ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
122742025-11-13T08:06:02.966ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
122752025-11-13T08:06:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
122762025-11-13T08:06:02.972ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
122772025-11-13T08:06:02.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
122782025-11-13T08:06:02.997ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122792025-11-13T08:06:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
122802025-11-13T08:06:03.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122812025-11-13T08:06:03.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122822025-11-13T08:06:03.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122832025-11-13T08:06:03.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122842025-11-13T08:06:03.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122852025-11-13T08:06:03.013ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
122862025-11-13T08:06:03.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
122872025-11-13T08:06:03.014ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
122882025-11-13T08:06:03.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122892025-11-13T08:06:03.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
122902025-11-13T08:06:03.016ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
122912025-11-13T08:06:03.016ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
122922025-11-13T08:06:03.018ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "4844d169-0e59-4bf2-849f-2f1fcc26af0b", "content-length": "81", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
122932025-11-13T08:06:03.019ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
122942025-11-13T08:06:03.019ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
122952025-11-13T08:06:03.020ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b187155-c159-4e4a-8091-4cca67880b92", "content-length": "81", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
122962025-11-13T08:06:03.022ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122972025-11-13T08:06:03.023ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 374.271008ms iteration = 15 status = {"disabled":false,"errors":[],"sps":[]}
122982025-11-13T08:06:03.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
122992025-11-13T08:06:03.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123002025-11-13T08:06:03.026ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
123012025-11-13T08:06:03.026ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
123022025-11-13T08:06:03.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
123032025-11-13T08:06:03.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123042025-11-13T08:06:03.029ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123052025-11-13T08:06:03.035ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
123062025-11-13T08:06:03.035ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
123072025-11-13T08:06:03.035ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
123082025-11-13T08:06:03.035ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
123092025-11-13T08:06:03.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123102025-11-13T08:06:03.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123112025-11-13T08:06:03.039ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123122025-11-13T08:06:03.044ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
123132025-11-13T08:06:03.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123142025-11-13T08:06:03.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123152025-11-13T08:06:03.045ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 437.612033ms iteration = 15 status = {}
123162025-11-13T08:06:03.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123172025-11-13T08:06:03.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123182025-11-13T08:06:03.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123192025-11-13T08:06:03.056ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
123202025-11-13T08:06:03.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123212025-11-13T08:06:03.057ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123222025-11-13T08:06:03.058ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
123232025-11-13T08:06:03.058ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 415.876656ms iteration = 8 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
123242025-11-13T08:06:03.059ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
123252025-11-13T08:06:03.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123262025-11-13T08:06:03.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123272025-11-13T08:06:03.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123282025-11-13T08:06:03.070ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 443.05717ms iteration = 24 status = {"physical_disks_added":0}
123292025-11-13T08:06:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
123302025-11-13T08:06:03.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123312025-11-13T08:06:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123322025-11-13T08:06:03.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123332025-11-13T08:06:03.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123342025-11-13T08:06:03.078ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123352025-11-13T08:06:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
123362025-11-13T08:06:03.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123372025-11-13T08:06:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123382025-11-13T08:06:03.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123392025-11-13T08:06:03.103ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123402025-11-13T08:06:03.108ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123412025-11-13T08:06:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123422025-11-13T08:06:03.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123432025-11-13T08:06:03.114ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
123442025-11-13T08:06:03.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123452025-11-13T08:06:03.115ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123462025-11-13T08:06:03.117ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123472025-11-13T08:06:03.120ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
123482025-11-13T08:06:03.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123492025-11-13T08:06:03.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123502025-11-13T08:06:03.121ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
123512025-11-13T08:06:03.122ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "928c62d0-e2b5-4dba-9a02-0c628db0d7fe", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
123522025-11-13T08:06:03.122ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
123532025-11-13T08:06:03.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123542025-11-13T08:06:03.124ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123552025-11-13T08:06:03.124ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "22d37695-4ee2-41df-bd94-1a8164615d28", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
123562025-11-13T08:06:03.124ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
123572025-11-13T08:06:03.125ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123582025-11-13T08:06:03.125ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123592025-11-13T08:06:03.126ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
123602025-11-13T08:06:03.126ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 519.368731ms iteration = 10 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
123612025-11-13T08:06:03.127ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123622025-11-13T08:06:03.132ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
123632025-11-13T08:06:03.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123642025-11-13T08:06:03.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123652025-11-13T08:06:03.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123662025-11-13T08:06:03.138ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
123672025-11-13T08:06:03.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123682025-11-13T08:06:03.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123692025-11-13T08:06:03.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123702025-11-13T08:06:03.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123712025-11-13T08:06:03.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123722025-11-13T08:06:03.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123732025-11-13T08:06:03.143ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123742025-11-13T08:06:03.144ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
123752025-11-13T08:06:03.144ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
123762025-11-13T08:06:03.144ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "20929f30-c1e5-439e-a484-bc2df43873df", "content-length": "372", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
123772025-11-13T08:06:03.144ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
123782025-11-13T08:06:03.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
123792025-11-13T08:06:03.147ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
123802025-11-13T08:06:03.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123812025-11-13T08:06:03.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123822025-11-13T08:06:03.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
123832025-11-13T08:06:03.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
123842025-11-13T08:06:03.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
123852025-11-13T08:06:03.154ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
123862025-11-13T08:06:03.175ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
123872025-11-13T08:06:03.177ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "d33f8eb3-c3f8-404a-b313-8579fddfa12b", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
123882025-11-13T08:06:03.179ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = FirewallRules step id = Ensure
123892025-11-13T08:06:03.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123902025-11-13T08:06:03.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123912025-11-13T08:06:03.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123922025-11-13T08:06:03.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123932025-11-13T08:06:03.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123942025-11-13T08:06:03.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123952025-11-13T08:06:03.189ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123962025-11-13T08:06:03.189ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123972025-11-13T08:06:03.191ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
123982025-11-13T08:06:03.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
123992025-11-13T08:06:03.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124002025-11-13T08:06:03.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124012025-11-13T08:06:03.194ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124022025-11-13T08:06:03.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
124032025-11-13T08:06:03.200ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124042025-11-13T08:06:03.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124052025-11-13T08:06:03.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124062025-11-13T08:06:03.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124072025-11-13T08:06:03.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124082025-11-13T08:06:03.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124092025-11-13T08:06:03.205ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
124102025-11-13T08:06:03.205ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
124112025-11-13T08:06:03.205ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d41c7153-365b-4b54-a032-0305b4dea426", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
124122025-11-13T08:06:03.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124132025-11-13T08:06:03.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124142025-11-13T08:06:03.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124152025-11-13T08:06:03.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124162025-11-13T08:06:03.206ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6371, tv_nsec: 559144285 } })
124172025-11-13T08:06:03.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124182025-11-13T08:06:03.208ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124192025-11-13T08:06:03.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124202025-11-13T08:06:03.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124212025-11-13T08:06:03.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124222025-11-13T08:06:03.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124232025-11-13T08:06:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
124242025-11-13T08:06:03.211ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124252025-11-13T08:06:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124262025-11-13T08:06:03.212ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124272025-11-13T08:06:03.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124282025-11-13T08:06:03.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124292025-11-13T08:06:03.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124302025-11-13T08:06:03.214ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
124312025-11-13T08:06:03.214ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124322025-11-13T08:06:03.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124332025-11-13T08:06:03.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124342025-11-13T08:06:03.215ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
124352025-11-13T08:06:03.215ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
124362025-11-13T08:06:03.215ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124372025-11-13T08:06:03.216ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "97f6e520-50e9-46a1-b5a3-5383620867fd", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
124382025-11-13T08:06:03.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124392025-11-13T08:06:03.217ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
124402025-11-13T08:06:03.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124412025-11-13T08:06:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124422025-11-13T08:06:03.217ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124432025-11-13T08:06:03.218ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "989c6b2a-b324-411a-9782-69aac025d9a0", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
124442025-11-13T08:06:03.218ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
124452025-11-13T08:06:03.218ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
124462025-11-13T08:06:03.219ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "10c63c55-248b-4c7f-903a-9eefcfcb3a8d", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
124472025-11-13T08:06:03.219ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
124482025-11-13T08:06:03.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124492025-11-13T08:06:03.219ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 577.402211ms iteration = 22 status = {}
124502025-11-13T08:06:03.220ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124512025-11-13T08:06:03.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124522025-11-13T08:06:03.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124532025-11-13T08:06:03.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124542025-11-13T08:06:03.222ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
124552025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 573.221219ms iteration = 8 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
124562025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 23 reason = Dependency
124572025-11-13T08:06:03.222ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
124582025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124592025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124602025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
124612025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124622025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124632025-11-13T08:06:03.222ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "de52192c-0ea2-4aed-aefd-b18de51bc684", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
124642025-11-13T08:06:03.222ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
124652025-11-13T08:06:03.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124662025-11-13T08:06:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124672025-11-13T08:06:03.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124682025-11-13T08:06:03.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124692025-11-13T08:06:03.224ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124702025-11-13T08:06:03.224ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "41f7211b-1686-4af9-bdb9-dcdac8c6734a", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
124712025-11-13T08:06:03.224ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
124722025-11-13T08:06:03.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124732025-11-13T08:06:03.226ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
124742025-11-13T08:06:03.226ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124752025-11-13T08:06:03.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124762025-11-13T08:06:03.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124772025-11-13T08:06:03.227ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
124782025-11-13T08:06:03.227ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 619.616241ms iteration = 10 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
124792025-11-13T08:06:03.227ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124802025-11-13T08:06:03.230ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
124812025-11-13T08:06:03.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124822025-11-13T08:06:03.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124832025-11-13T08:06:03.231ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
124842025-11-13T08:06:03.231ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
124852025-11-13T08:06:03.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
124862025-11-13T08:06:03.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124872025-11-13T08:06:03.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124882025-11-13T08:06:03.232ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
124892025-11-13T08:06:03.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124902025-11-13T08:06:03.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124912025-11-13T08:06:03.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124922025-11-13T08:06:03.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124932025-11-13T08:06:03.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
124942025-11-13T08:06:03.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
124952025-11-13T08:06:03.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124962025-11-13T08:06:03.235ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
124972025-11-13T08:06:03.235ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124982025-11-13T08:06:03.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
124992025-11-13T08:06:03.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125002025-11-13T08:06:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
125012025-11-13T08:06:03.238ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
125022025-11-13T08:06:03.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
125032025-11-13T08:06:03.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125042025-11-13T08:06:03.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125052025-11-13T08:06:03.240ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125062025-11-13T08:06:03.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125072025-11-13T08:06:03.242ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125082025-11-13T08:06:03.248ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
125092025-11-13T08:06:03.248ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125102025-11-13T08:06:03.248ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125112025-11-13T08:06:03.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125122025-11-13T08:06:03.250ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125132025-11-13T08:06:03.251ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125142025-11-13T08:06:03.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125152025-11-13T08:06:03.253ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125162025-11-13T08:06:03.258ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
125172025-11-13T08:06:03.258ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125182025-11-13T08:06:03.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125192025-11-13T08:06:03.259ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
125202025-11-13T08:06:03.259ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "36c97629-72a6-410b-a646-3f549826777a", "content-length": "691", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
125212025-11-13T08:06:03.259ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
125222025-11-13T08:06:03.259ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
125232025-11-13T08:06:03.259ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
125242025-11-13T08:06:03.262ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 619.413518ms iteration = 15 status = {}
125252025-11-13T08:06:03.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125262025-11-13T08:06:03.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125272025-11-13T08:06:03.264ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125282025-11-13T08:06:03.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125292025-11-13T08:06:03.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125302025-11-13T08:06:03.266ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125312025-11-13T08:06:03.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125322025-11-13T08:06:03.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125332025-11-13T08:06:03.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125342025-11-13T08:06:03.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125352025-11-13T08:06:03.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125362025-11-13T08:06:03.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125372025-11-13T08:06:03.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125382025-11-13T08:06:03.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125392025-11-13T08:06:03.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125402025-11-13T08:06:03.280ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
125412025-11-13T08:06:03.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125422025-11-13T08:06:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125432025-11-13T08:06:03.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125442025-11-13T08:06:03.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125452025-11-13T08:06:03.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125462025-11-13T08:06:03.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125472025-11-13T08:06:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125482025-11-13T08:06:03.290ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
125492025-11-13T08:06:03.290ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
125502025-11-13T08:06:03.290ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 911e1030-f3ab-42af-98e8-6157fda248e3 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
125512025-11-13T08:06:03.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
125522025-11-13T08:06:03.290ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
125532025-11-13T08:06:03.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125542025-11-13T08:06:03.293ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
125552025-11-13T08:06:03.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125562025-11-13T08:06:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125572025-11-13T08:06:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
125582025-11-13T08:06:03.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
125592025-11-13T08:06:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
125602025-11-13T08:06:03.299ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
125612025-11-13T08:06:03.299ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
125622025-11-13T08:06:03.299ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 680.897237ms iteration = 19 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
125632025-11-13T08:06:03.300ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
125642025-11-13T08:06:03.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125652025-11-13T08:06:03.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125662025-11-13T08:06:03.302ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125672025-11-13T08:06:03.303ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
125682025-11-13T08:06:03.303ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
125692025-11-13T08:06:03.303ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a67202e-f76f-493f-85b6-d5435b95a2bf", "content-length": "372", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
125702025-11-13T08:06:03.304ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
125712025-11-13T08:06:03.304ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125722025-11-13T08:06:03.305ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
125732025-11-13T08:06:03.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
125742025-11-13T08:06:03.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
125752025-11-13T08:06:03.306ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
125762025-11-13T08:06:03.306ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
125772025-11-13T08:06:03.306ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
125782025-11-13T08:06:03.320ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125792025-11-13T08:06:03.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125802025-11-13T08:06:03.322ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
125812025-11-13T08:06:03.322ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
125822025-11-13T08:06:03.322ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "6d00c145-4a2a-410e-9e9b-3d9ebaa27d73", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
125832025-11-13T08:06:03.322ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
125842025-11-13T08:06:03.322ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125852025-11-13T08:06:03.322ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
125862025-11-13T08:06:03.322ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
125872025-11-13T08:06:03.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
125882025-11-13T08:06:03.324ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
125892025-11-13T08:06:03.324ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125902025-11-13T08:06:03.327ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
125912025-11-13T08:06:03.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
125922025-11-13T08:06:03.330ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
125932025-11-13T08:06:03.330ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
125942025-11-13T08:06:03.330ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "81ec1ae3-35fa-4674-aa9f-6e38586f2e9c", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
125952025-11-13T08:06:03.332ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 109.84316ms iteration = 23 status = {}
125962025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
125972025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
125982025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
125992025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
126002025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
126012025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
126022025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
126032025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
126042025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
126052025-11-13T08:06:03.334ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
126062025-11-13T08:06:03.335ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
126072025-11-13T08:06:03.335ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
126082025-11-13T08:06:03.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
126092025-11-13T08:06:03.337ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
126102025-11-13T08:06:03.337ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
126112025-11-13T08:06:03.371ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
126122025-11-13T08:06:03.371ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
126132025-11-13T08:06:03.403ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126142025-11-13T08:06:03.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126152025-11-13T08:06:03.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126162025-11-13T08:06:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126172025-11-13T08:06:03.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126182025-11-13T08:06:03.410ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
126192025-11-13T08:06:03.410ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
126202025-11-13T08:06:03.418ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "a2b08c82-f1ce-4468-8100-576708069e0c", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
126212025-11-13T08:06:03.419ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
126222025-11-13T08:06:03.419ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = Dns step id = Ensure
126232025-11-13T08:06:03.419ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
126242025-11-13T08:06:03.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126252025-11-13T08:06:03.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126262025-11-13T08:06:03.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126272025-11-13T08:06:03.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126282025-11-13T08:06:03.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126292025-11-13T08:06:03.426ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126302025-11-13T08:06:03.427ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
126312025-11-13T08:06:03.427ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
126322025-11-13T08:06:03.427ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
126332025-11-13T08:06:03.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126342025-11-13T08:06:03.427ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126352025-11-13T08:06:03.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126362025-11-13T08:06:03.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126372025-11-13T08:06:03.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126382025-11-13T08:06:03.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126392025-11-13T08:06:03.468ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
126402025-11-13T08:06:03.468ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
126412025-11-13T08:06:03.468ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126422025-11-13T08:06:03.468ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126432025-11-13T08:06:03.470ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126442025-11-13T08:06:03.473ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126452025-11-13T08:06:03.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126462025-11-13T08:06:03.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126472025-11-13T08:06:03.473ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "d8c9dd83-4706-4efc-9eae-6e71be166a06", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
126482025-11-13T08:06:03.473ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
126492025-11-13T08:06:03.473ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
126502025-11-13T08:06:03.476ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
126512025-11-13T08:06:03.476ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
126522025-11-13T08:06:03.476ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
126532025-11-13T08:06:03.476ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126542025-11-13T08:06:03.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126552025-11-13T08:06:03.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126562025-11-13T08:06:03.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126572025-11-13T08:06:03.478ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126582025-11-13T08:06:03.480ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1094fb8-4dfa-4dfa-8189-8fc08ed99b90", "content-length": "682", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
126592025-11-13T08:06:03.480ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
126602025-11-13T08:06:03.481ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
126612025-11-13T08:06:03.482ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126622025-11-13T08:06:03.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
126632025-11-13T08:06:03.484ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
126642025-11-13T08:06:03.489ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
126652025-11-13T08:06:03.489ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
126662025-11-13T08:06:03.491ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
126672025-11-13T08:06:03.492ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
126682025-11-13T08:06:03.504ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 861.955955ms iteration = 15 status = {}
126692025-11-13T08:06:03.526ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
126702025-11-13T08:06:03.526ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
126712025-11-13T08:06:03.526ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
126722025-11-13T08:06:03.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126732025-11-13T08:06:03.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126742025-11-13T08:06:03.528ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126752025-11-13T08:06:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126762025-11-13T08:06:03.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126772025-11-13T08:06:03.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126782025-11-13T08:06:03.534ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
126792025-11-13T08:06:03.534ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
126802025-11-13T08:06:03.534ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126812025-11-13T08:06:03.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126822025-11-13T08:06:03.535ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126832025-11-13T08:06:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
126842025-11-13T08:06:03.538ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126852025-11-13T08:06:03.538ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126862025-11-13T08:06:03.540ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
126872025-11-13T08:06:03.540ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
126882025-11-13T08:06:03.541ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126892025-11-13T08:06:03.541ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126902025-11-13T08:06:03.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126912025-11-13T08:06:03.544ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
126922025-11-13T08:06:03.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126932025-11-13T08:06:03.544ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126942025-11-13T08:06:03.546ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126952025-11-13T08:06:03.546ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126962025-11-13T08:06:03.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
126972025-11-13T08:06:03.547ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
126982025-11-13T08:06:03.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126992025-11-13T08:06:03.551ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
127002025-11-13T08:06:03.553ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
127012025-11-13T08:06:03.553ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
127022025-11-13T08:06:03.553ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = OmicronZones step id = Cleanup
127032025-11-13T08:06:03.553ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = OmicronZones step id = Cleanup
127042025-11-13T08:06:03.553ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = PhysicalDisks step id = Cleanup
127052025-11-13T08:06:03.553ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = Clickhouse step id = Ensure
127062025-11-13T08:06:03.553ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = Clickhouse step id = Ensure
127072025-11-13T08:06:03.583ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
127082025-11-13T08:06:03.588ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "418520c9-55f4-456a-aaac-9215d2544b8a", "date": "Thu, 13 Nov 2025 08:06:03 GMT"} })
127092025-11-13T08:06:03.589ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = SupportBundles step id = Cleanup
127102025-11-13T08:06:03.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
127112025-11-13T08:06:03.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
127122025-11-13T08:06:03.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127132025-11-13T08:06:03.592ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
127142025-11-13T08:06:03.592ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
127152025-11-13T08:06:03.592ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
127162025-11-13T08:06:03.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
127172025-11-13T08:06:03.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
127182025-11-13T08:06:03.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127192025-11-13T08:06:03.602ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
127202025-11-13T08:06:03.640ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = OmicronZones step id = Cleanup
127212025-11-13T08:06:03.640ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
127222025-11-13T08:06:03.640ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
127232025-11-13T08:06:03.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
127242025-11-13T08:06:03.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
127252025-11-13T08:06:03.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127262025-11-13T08:06:03.643ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
127272025-11-13T08:06:03.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
127282025-11-13T08:06:03.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
127292025-11-13T08:06:03.679ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
127302025-11-13T08:06:03.679ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
127312025-11-13T08:06:03.679ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = Cockroach step id = Ensure
127322025-11-13T08:06:03.679ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
127332025-11-13T08:06:03.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
127342025-11-13T08:06:03.690ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
127352025-11-13T08:06:03.690ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 1985f5ff-0bd0-401d-9458-e629affe97da step component = MgsUpdates step id = Ensure
127362025-11-13T08:06:03.690ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 23 reason = Dependency
127372025-11-13T08:06:03.690ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 9 reason = Dependency
127382025-11-13T08:06:03.690ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127392025-11-13T08:06:03.690ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
127402025-11-13T08:06:03.690ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.082858058s iteration = 9 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"7abd9b88-d191-4580-ad62-02272311b722","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":4520,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":138056894,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":138056844,"secs":0}},"event_index":1,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":138083475,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":65542789,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":65542729,"secs":0}},"event_index":2,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":203655284,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":102513705,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":102513645,"secs":0}},"event_index":3,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":306201450,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":265020120,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":265020080,"secs":0}},"event_index":4,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":571249011,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":240644041,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":240644001,"secs":0}},"event_index":5,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":811907412,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":133699738,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":133699708,"secs":0}},"event_index":6,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":945624260,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3770,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3740,"secs":0}},"event_index":7,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":945642451,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2130,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2090,"secs":0}},"event_index":8,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":945647651,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2970,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2930,"secs":0}},"event_index":9,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":945652331,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1790,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1750,"secs":0}},"event_index":10,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":945655701,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35550669,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":35550629,"secs":0}},"event_index":11,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":981207370,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":50908591,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":50908561,"secs":0}},"event_index":12,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":32126321,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39576369,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":39576329,"secs":0}},"event_index":13,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":71721529,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":10668659,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":10668619,"secs":0}},"event_index":14,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":82403710,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5780,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5740,"secs":0}},"event_index":15,"execution_id":"7abd9b88-d191-4580-ad62-02272311b722","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":82418670,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
127412025-11-13T08:06:03.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127422025-11-13T08:06:03.690ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
127432025-11-13T08:06:03.692ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127442025-11-13T08:06:03.692ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127452025-11-13T08:06:03.695ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
127462025-11-13T08:06:03.695ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
127472025-11-13T08:06:03.695ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
127482025-11-13T08:06:03.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
127492025-11-13T08:06:03.695ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127502025-11-13T08:06:03.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127512025-11-13T08:06:03.703ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a, 0), (64225556-e244-4e8f-b528-cc1f9cf4dea6, 0), (9ef70888-af19-4061-b38f-7f8e1ee65d5f, 0), (54ba8229-42c5-4cf8-a019-8fd3109d6083, 0)] collection_id = a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a
127522025-11-13T08:06:03.703ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127532025-11-13T08:06:03.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127542025-11-13T08:06:03.738ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
127552025-11-13T08:06:03.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
127562025-11-13T08:06:03.740ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127572025-11-13T08:06:03.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
127582025-11-13T08:06:03.742ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
127592025-11-13T08:06:03.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
127602025-11-13T08:06:03.779ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
127612025-11-13T08:06:03.779ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 88.610122ms iteration = 23 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
127622025-11-13T08:06:03.836ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = a0fdbf4e-da2b-4a51-8ae7-ee7360c1df7a file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
127632025-11-13T08:06:03.836ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127642025-11-13T08:06:03.836ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127652025-11-13T08:06:03.845ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(64225556-e244-4e8f-b528-cc1f9cf4dea6, 0), (9ef70888-af19-4061-b38f-7f8e1ee65d5f, 0), (54ba8229-42c5-4cf8-a019-8fd3109d6083, 0), (14f5dd9e-f757-4845-8f22-abf43341e043, 0)] collection_id = 64225556-e244-4e8f-b528-cc1f9cf4dea6
127662025-11-13T08:06:03.845ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127672025-11-13T08:06:03.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127682025-11-13T08:06:03.977ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 45 transaction = inventory_delete_collection
127692025-11-13T08:06:04.059ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
127702025-11-13T08:06:04.059ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 64225556-e244-4e8f-b528-cc1f9cf4dea6 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
127712025-11-13T08:06:04.059ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127722025-11-13T08:06:04.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
127732025-11-13T08:06:04.103ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(54ba8229-42c5-4cf8-a019-8fd3109d6083, 0), (14f5dd9e-f757-4845-8f22-abf43341e043, 0), (c3f9dc97-7cb9-4750-a624-009dc8a84400, 0)]
127742025-11-13T08:06:04.104ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6372, tv_nsec: 456663219 } })
127752025-11-13T08:06:04.136ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6372, tv_nsec: 489045336 } })
127762025-11-13T08:06:04.137ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
127772025-11-13T08:06:04.137ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
127782025-11-13T08:06:04.137ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
127792025-11-13T08:06:04.137ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "386a2135-a958-4fa7-9cde-bc449c2d16f7", "content-length": "476", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127802025-11-13T08:06:04.137ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
127812025-11-13T08:06:04.138ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f1a7bc3-4b75-4e1a-a928-834ee701a2af", "content-length": "734", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127822025-11-13T08:06:04.138ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
127832025-11-13T08:06:04.138ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff8be824-ac5f-4179-9250-09df6319fd46", "content-length": "111", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127842025-11-13T08:06:04.138ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
127852025-11-13T08:06:04.139ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6358cdef-911b-45b3-8e63-c49c99228e35", "content-length": "111", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127862025-11-13T08:06:04.139ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
127872025-11-13T08:06:04.139ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5181a94d-1fd5-49ee-9acd-7d51c1498604", "content-length": "163", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127882025-11-13T08:06:04.139ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
127892025-11-13T08:06:04.139ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4099fb37-71d4-4279-b274-5564e9108f9d", "content-length": "163", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127902025-11-13T08:06:04.139ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
127912025-11-13T08:06:04.140ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "920ba6f8-3cfc-4330-b6a0-d72930a67064", "content-length": "166", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127922025-11-13T08:06:04.140ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
127932025-11-13T08:06:04.140ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "09f0223f-834d-4646-b02a-08fb40d6d59f", "content-length": "166", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127942025-11-13T08:06:04.140ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
127952025-11-13T08:06:04.140ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecc4f90c-b4db-45b7-b927-13addba9df6e", "content-length": "702", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127962025-11-13T08:06:04.140ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
127972025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5f3f786-c132-402f-a5a7-966d3bcbdb85", "content-length": "727", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
127982025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
127992025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e0dc4b8-f186-4767-8e19-98c202896748", "content-length": "729", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128002025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
128012025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dece39dd-cda7-49bf-8f41-182a4f9fede4", "content-length": "728", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128022025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
128032025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "234888a5-b766-4ab9-bf84-4d554efa2a28", "content-length": "722", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128042025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
128052025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "53382e21-4cee-4c32-b9e4-b4bc8d090211", "content-length": "10", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128062025-11-13T08:06:04.141ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
128072025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c0d89dd-fb73-43bb-a3e2-77d09f0bedcc", "content-length": "144", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128082025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
128092025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa28afe2-92d2-44ca-8b43-8f7eb02d91b8", "content-length": "147", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128102025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
128112025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e323d973-5304-4461-8b0c-174b49a382e5", "content-length": "109", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128122025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
128132025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98c5901d-a8d7-491e-806b-4de4ff4289fe", "content-length": "109", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128142025-11-13T08:06:04.142ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
128152025-11-13T08:06:04.143ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6051b293-d91f-4e9c-9de7-aab791a29a8c", "content-length": "163", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128162025-11-13T08:06:04.143ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
128172025-11-13T08:06:04.143ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc92332f-fef2-4e6e-a60f-cc72e80f5729", "content-length": "163", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128182025-11-13T08:06:04.143ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
128192025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8898e565-476e-4f84-af95-100d68c73a0f", "content-length": "166", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128202025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
128212025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f88c8a25-1352-432a-86da-215173d672ed", "content-length": "166", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128222025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
128232025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eda9743-65a8-4904-b58a-879d15e4aa48", "content-length": "702", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128242025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
128252025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4341ca8-5c39-4e2a-99ae-22286e4a0a86", "content-length": "727", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128262025-11-13T08:06:04.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
128272025-11-13T08:06:04.145ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bf74bc4-4a55-413a-b400-cc39f4037564", "content-length": "729", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128282025-11-13T08:06:04.145ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
128292025-11-13T08:06:04.145ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cce2138d-507e-4615-82a1-b0a734174b26", "content-length": "728", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128302025-11-13T08:06:04.145ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128312025-11-13T08:06:04.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
128322025-11-13T08:06:04.147ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128332025-11-13T08:06:04.149ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
128342025-11-13T08:06:04.149ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128352025-11-13T08:06:04.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
128362025-11-13T08:06:04.151ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128372025-11-13T08:06:04.153ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
128382025-11-13T08:06:04.153ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128392025-11-13T08:06:04.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
128402025-11-13T08:06:04.185ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
128412025-11-13T08:06:04.185ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
128422025-11-13T08:06:04.240ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0cb6a93-752a-431e-ab12-7b12b5dcdb6a", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128432025-11-13T08:06:04.243ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
128442025-11-13T08:06:04.243ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
128452025-11-13T08:06:04.243ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
128462025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
128472025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
128482025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
128492025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
128502025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
128512025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9affcf3-db08-43a9-b0b6-c6bc200fbf46", "content-length": "64", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128522025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
128532025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ed84df9-4732-4bfa-8a6b-845dd3bc1201", "content-length": "64", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128542025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0edf687-946b-4bc7-b37f-83d1034ddce3", "content-length": "64", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128552025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
128562025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2c37346-be3e-4b24-a4cd-de9a2549c5ac", "content-length": "64", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128572025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
128582025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0457dc5-96e1-4976-932e-3b16176abe24", "content-length": "64", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128592025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
128602025-11-13T08:06:04.243ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
128612025-11-13T08:06:04.283ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aa240a8-92d1-4a39-8b9e-b6d4d23fbbcd", "content-length": "396847", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128622025-11-13T08:06:04.284ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "39dcff82-c486-4401-8f22-2f77dfbe6805", "content-length": "421694", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128632025-11-13T08:06:04.284ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "16e0e920-d6d0-4c70-8d88-4f7384e52f8b", "content-length": "396344", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128642025-11-13T08:06:04.285ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "715da008-a5d3-4415-9e8a-12a03d732e24", "content-length": "404369", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128652025-11-13T08:06:04.285ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f45c4fc-839c-42e7-97b5-52ce091691c5", "content-length": "393583", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128662025-11-13T08:06:04.288ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
128672025-11-13T08:06:04.288ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
128682025-11-13T08:06:04.288ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
128692025-11-13T08:06:04.288ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
128702025-11-13T08:06:04.288ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
128712025-11-13T08:06:04.324ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
128722025-11-13T08:06:04.324ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
128732025-11-13T08:06:04.328ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "10aab996-74e3-4e16-81de-6c1fd995d5ab", "content-length": "124", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128742025-11-13T08:06:04.329ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
128752025-11-13T08:06:04.419ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
128762025-11-13T08:06:04.419ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
128772025-11-13T08:06:04.420ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6436ccef-352d-4ea7-a87d-8f5464091c13", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128782025-11-13T08:06:04.420ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
128792025-11-13T08:06:04.422ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
128802025-11-13T08:06:04.422ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
128812025-11-13T08:06:04.422ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d89c6221-dfb1-451b-946c-7d85fb610113", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128822025-11-13T08:06:04.422ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
128832025-11-13T08:06:04.424ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
128842025-11-13T08:06:04.424ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
128852025-11-13T08:06:04.425ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "652e2f64-1dee-42b3-bb0b-dd55a2cdf835", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:06:04 GMT"} })
128862025-11-13T08:06:04.425ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
128872025-11-13T08:06:04.427ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
128882025-11-13T08:06:04.427ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
128892025-11-13T08:06:04.429ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128902025-11-13T08:06:04.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
128912025-11-13T08:06:04.431ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128922025-11-13T08:06:04.433ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
128932025-11-13T08:06:04.434ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128942025-11-13T08:06:04.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
128952025-11-13T08:06:04.630ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = db0127fd-bd52-4625-be88-6185cc000069 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
128962025-11-13T08:06:04.630ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = db0127fd-bd52-4625-be88-6185cc000069 time_started = 2025-11-13 08:06:04.137328 UTC
128972025-11-13T08:06:04.630ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 38 reason = Dependency
128982025-11-13T08:06:04.630ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
128992025-11-13T08:06:04.630ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 940.05671ms iteration = 9 status = {"collection_id":"db0127fd-bd52-4625-be88-6185cc000069","time_done":"2025-11-13 08:06:04.427142 UTC","time_started":"2025-11-13 08:06:04.137328 UTC"}
129002025-11-13T08:06:04.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
129012025-11-13T08:06:04.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129022025-11-13T08:06:04.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
129032025-11-13T08:06:04.636ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
129042025-11-13T08:06:04.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
129052025-11-13T08:06:04.770ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
129062025-11-13T08:06:04.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
129072025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 188.645208ms iteration = 38 status = {"Loaded":{"collection_id":"4ba91f9a-a9b4-4c47-93bd-d51c608704f2","time_loaded":"2025-11-13T08:06:04.630455467Z","time_started":"2025-11-13T08:06:04.188335Z"}}
129082025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 12 reason = Dependency
129092025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 20 reason = Dependency
129102025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 25 reason = Dependency
129112025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129122025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129132025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129142025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129152025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129162025-11-13T08:06:04.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129172025-11-13T08:06:04.821ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129182025-11-13T08:06:04.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129192025-11-13T08:06:04.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129202025-11-13T08:06:04.824ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
129212025-11-13T08:06:04.824ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129222025-11-13T08:06:04.824ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
129232025-11-13T08:06:04.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129242025-11-13T08:06:04.824ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
129252025-11-13T08:06:04.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129262025-11-13T08:06:04.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129272025-11-13T08:06:04.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129282025-11-13T08:06:04.825ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129292025-11-13T08:06:04.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129302025-11-13T08:06:04.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129312025-11-13T08:06:04.827ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129322025-11-13T08:06:04.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129332025-11-13T08:06:04.828ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129342025-11-13T08:06:04.829ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
129352025-11-13T08:06:04.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129362025-11-13T08:06:04.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129372025-11-13T08:06:04.829ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
129382025-11-13T08:06:04.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129392025-11-13T08:06:04.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129402025-11-13T08:06:04.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129412025-11-13T08:06:04.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129422025-11-13T08:06:04.831ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
129432025-11-13T08:06:04.831ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
129442025-11-13T08:06:04.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129452025-11-13T08:06:04.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
129462025-11-13T08:06:04.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129472025-11-13T08:06:04.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129482025-11-13T08:06:04.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129492025-11-13T08:06:04.837ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
129502025-11-13T08:06:04.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129512025-11-13T08:06:04.837ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129522025-11-13T08:06:04.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129532025-11-13T08:06:04.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
129542025-11-13T08:06:04.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129552025-11-13T08:06:04.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129562025-11-13T08:06:04.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129572025-11-13T08:06:04.841ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
129582025-11-13T08:06:04.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129592025-11-13T08:06:04.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129602025-11-13T08:06:04.842ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 23.742607ms iteration = 25 status = {"physical_disks_added":0}
129612025-11-13T08:06:04.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
129622025-11-13T08:06:04.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129632025-11-13T08:06:04.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129642025-11-13T08:06:04.844ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
129652025-11-13T08:06:04.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129662025-11-13T08:06:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129672025-11-13T08:06:04.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129682025-11-13T08:06:04.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129692025-11-13T08:06:04.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129702025-11-13T08:06:04.848ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
129712025-11-13T08:06:04.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129722025-11-13T08:06:04.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129732025-11-13T08:06:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129742025-11-13T08:06:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129752025-11-13T08:06:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129762025-11-13T08:06:04.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129772025-11-13T08:06:04.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
129782025-11-13T08:06:04.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
129792025-11-13T08:06:04.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
129802025-11-13T08:06:04.854ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
129812025-11-13T08:06:04.854ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
129822025-11-13T08:06:04.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129832025-11-13T08:06:04.854ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 34.949797ms iteration = 12 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"4ba91f9a-a9b4-4c47-93bd-d51c608704f2","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
129842025-11-13T08:06:04.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129852025-11-13T08:06:04.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129862025-11-13T08:06:04.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129872025-11-13T08:06:04.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129882025-11-13T08:06:04.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
129892025-11-13T08:06:04.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129902025-11-13T08:06:04.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129912025-11-13T08:06:04.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129922025-11-13T08:06:04.865ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
129932025-11-13T08:06:04.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129942025-11-13T08:06:04.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129952025-11-13T08:06:04.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
129962025-11-13T08:06:04.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
129972025-11-13T08:06:04.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129982025-11-13T08:06:04.874ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
129992025-11-13T08:06:04.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130002025-11-13T08:06:04.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130012025-11-13T08:06:04.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130022025-11-13T08:06:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130032025-11-13T08:06:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130042025-11-13T08:06:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
130052025-11-13T08:06:04.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130062025-11-13T08:06:04.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130072025-11-13T08:06:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130082025-11-13T08:06:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130092025-11-13T08:06:04.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130102025-11-13T08:06:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
130112025-11-13T08:06:04.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130122025-11-13T08:06:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130132025-11-13T08:06:04.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130142025-11-13T08:06:04.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130152025-11-13T08:06:04.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130162025-11-13T08:06:04.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130172025-11-13T08:06:04.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130182025-11-13T08:06:04.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
130192025-11-13T08:06:04.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130202025-11-13T08:06:04.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130212025-11-13T08:06:04.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130222025-11-13T08:06:04.914ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
130232025-11-13T08:06:04.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130242025-11-13T08:06:04.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130252025-11-13T08:06:04.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130262025-11-13T08:06:04.917ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130272025-11-13T08:06:04.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130282025-11-13T08:06:04.919ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130292025-11-13T08:06:04.920ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130302025-11-13T08:06:04.923ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
130312025-11-13T08:06:04.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130322025-11-13T08:06:04.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130332025-11-13T08:06:04.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130342025-11-13T08:06:04.927ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
130352025-11-13T08:06:04.927ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130362025-11-13T08:06:04.927ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130372025-11-13T08:06:04.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130382025-11-13T08:06:04.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130392025-11-13T08:06:04.930ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130402025-11-13T08:06:04.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130412025-11-13T08:06:04.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130422025-11-13T08:06:04.935ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
130432025-11-13T08:06:04.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130442025-11-13T08:06:04.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130452025-11-13T08:06:04.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130462025-11-13T08:06:04.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130472025-11-13T08:06:04.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130482025-11-13T08:06:04.940ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
130492025-11-13T08:06:04.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130502025-11-13T08:06:04.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130512025-11-13T08:06:04.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130522025-11-13T08:06:04.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130532025-11-13T08:06:04.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130542025-11-13T08:06:04.947ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
130552025-11-13T08:06:04.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130562025-11-13T08:06:04.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130572025-11-13T08:06:04.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130582025-11-13T08:06:04.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130592025-11-13T08:06:04.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130602025-11-13T08:06:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
130612025-11-13T08:06:04.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130622025-11-13T08:06:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130632025-11-13T08:06:04.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130642025-11-13T08:06:04.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130652025-11-13T08:06:04.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130662025-11-13T08:06:04.960ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
130672025-11-13T08:06:04.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130682025-11-13T08:06:04.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130692025-11-13T08:06:04.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130702025-11-13T08:06:04.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130712025-11-13T08:06:04.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130722025-11-13T08:06:04.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
130732025-11-13T08:06:04.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130742025-11-13T08:06:04.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130752025-11-13T08:06:05.000ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
130762025-11-13T08:06:05.000ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
130772025-11-13T08:06:05.000ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 931c2060-42ef-4991-892c-3bfed6c6173d parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
130782025-11-13T08:06:05.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130792025-11-13T08:06:05.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130802025-11-13T08:06:05.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130812025-11-13T08:06:05.004ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
130822025-11-13T08:06:05.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
130832025-11-13T08:06:05.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
130842025-11-13T08:06:05.007ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
130852025-11-13T08:06:05.007ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
130862025-11-13T08:06:05.007ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 188.67957ms iteration = 20 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
130872025-11-13T08:06:05.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 459 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 24d03856-0005-4507-96ce-ae8c96ebf2ae response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
130882025-11-13T08:06:07.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 86 reason = Timeout
130892025-11-13T08:06:07.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130902025-11-13T08:06:07.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130912025-11-13T08:06:07.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130922025-11-13T08:06:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
130932025-11-13T08:06:07.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
130942025-11-13T08:06:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
130952025-11-13T08:06:07.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
130962025-11-13T08:06:07.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.485157ms iteration = 86 status = {"config_updated":false}
130972025-11-13T08:06:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 44 reason = Timeout
130982025-11-13T08:06:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 87 reason = Timeout
130992025-11-13T08:06:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131002025-11-13T08:06:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131012025-11-13T08:06:12.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131022025-11-13T08:06:12.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131032025-11-13T08:06:12.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131042025-11-13T08:06:12.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131052025-11-13T08:06:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
131062025-11-13T08:06:12.633ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131072025-11-13T08:06:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131082025-11-13T08:06:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
131092025-11-13T08:06:12.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131102025-11-13T08:06:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131112025-11-13T08:06:12.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131122025-11-13T08:06:12.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131132025-11-13T08:06:12.635ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
131142025-11-13T08:06:12.635ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.502667ms iteration = 87 status = {"config_updated":false}
131152025-11-13T08:06:12.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131162025-11-13T08:06:12.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
131172025-11-13T08:06:12.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131182025-11-13T08:06:12.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131192025-11-13T08:06:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 44 reason = Timeout
131202025-11-13T08:06:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131212025-11-13T08:06:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131222025-11-13T08:06:12.655ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
131232025-11-13T08:06:12.655ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.771835ms iteration = 44 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
131242025-11-13T08:06:12.689ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131252025-11-13T08:06:12.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131262025-11-13T08:06:12.691ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 49.056705ms iteration = 44 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
131272025-11-13T08:06:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 268 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 69be5176-b2e8-4727-9481-11c7d014da86 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
131282025-11-13T08:06:17.625ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 39 reason = Timeout
131292025-11-13T08:06:17.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 88 reason = Timeout
131302025-11-13T08:06:17.625ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
131312025-11-13T08:06:17.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131322025-11-13T08:06:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
131332025-11-13T08:06:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131342025-11-13T08:06:17.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131352025-11-13T08:06:17.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131362025-11-13T08:06:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
131372025-11-13T08:06:17.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131382025-11-13T08:06:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131392025-11-13T08:06:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
131402025-11-13T08:06:17.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
131412025-11-13T08:06:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
131422025-11-13T08:06:17.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
131432025-11-13T08:06:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.719212ms iteration = 88 status = {"config_updated":false}
131442025-11-13T08:06:17.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4ba91f9a-a9b4-4c47-93bd-d51c608704f2 original_time_started = 2025-11-13 08:06:04.188335 UTC
131452025-11-13T08:06:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.823013ms iteration = 39 status = {"Loaded":{"collection_id":"4ba91f9a-a9b4-4c47-93bd-d51c608704f2","time_loaded":"2025-11-13T08:06:17.625928777Z","time_started":"2025-11-13T08:06:04.188335Z"}}
131462025-11-13T08:06:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 30 reason = Timeout
131472025-11-13T08:06:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 30 reason = Timeout
131482025-11-13T08:06:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
131492025-11-13T08:06:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
131502025-11-13T08:06:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
131512025-11-13T08:06:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
131522025-11-13T08:06:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
131532025-11-13T08:06:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
131542025-11-13T08:06:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131552025-11-13T08:06:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131562025-11-13T08:06:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
131572025-11-13T08:06:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
131582025-11-13T08:06:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
131592025-11-13T08:06:17.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
131602025-11-13T08:06:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.432056ms iteration = 30 status = "NoSitrep"
131612025-11-13T08:06:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
131622025-11-13T08:06:17.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
131632025-11-13T08:06:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
131642025-11-13T08:06:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
131652025-11-13T08:06:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131662025-11-13T08:06:17.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
131672025-11-13T08:06:17.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
131682025-11-13T08:06:17.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.845853ms iteration = 30 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
131692025-11-13T08:06:22.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 45 reason = Timeout
131702025-11-13T08:06:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 89 reason = Timeout
131712025-11-13T08:06:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131722025-11-13T08:06:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131732025-11-13T08:06:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131742025-11-13T08:06:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131752025-11-13T08:06:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131762025-11-13T08:06:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131772025-11-13T08:06:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
131782025-11-13T08:06:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131792025-11-13T08:06:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
131802025-11-13T08:06:22.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
131812025-11-13T08:06:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131822025-11-13T08:06:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
131832025-11-13T08:06:22.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131842025-11-13T08:06:22.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
131852025-11-13T08:06:22.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.348294ms iteration = 89 status = {"config_updated":false}
131862025-11-13T08:06:22.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131872025-11-13T08:06:22.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131882025-11-13T08:06:22.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
131892025-11-13T08:06:22.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
131902025-11-13T08:06:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
131912025-11-13T08:06:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 45 reason = Timeout
131922025-11-13T08:06:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131932025-11-13T08:06:22.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131942025-11-13T08:06:22.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
131952025-11-13T08:06:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
131962025-11-13T08:06:22.655ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
131972025-11-13T08:06:22.655ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.773236ms iteration = 45 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
131982025-11-13T08:06:22.680ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.912495ms iteration = 45 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
131992025-11-13T08:06:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 283 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = d2f38494-059f-4afe-b036-d286b6b0d041 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
132002025-11-13T08:06:27.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 90 reason = Timeout
132012025-11-13T08:06:27.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
132022025-11-13T08:06:27.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
132032025-11-13T08:06:27.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132042025-11-13T08:06:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
132052025-11-13T08:06:27.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
132062025-11-13T08:06:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
132072025-11-13T08:06:27.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
132082025-11-13T08:06:27.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.239712ms iteration = 90 status = {"config_updated":false}
132092025-11-13T08:06:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 16 reason = Timeout
132102025-11-13T08:06:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 16 reason = Timeout
132112025-11-13T08:06:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 16 reason = Timeout
132122025-11-13T08:06:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
132132025-11-13T08:06:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
132142025-11-13T08:06:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
132152025-11-13T08:06:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
132162025-11-13T08:06:32.612ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6400, tv_nsec: 965235006 } })
132172025-11-13T08:06:32.613ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
132182025-11-13T08:06:32.617ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6400, tv_nsec: 969852907 } })
132192025-11-13T08:06:32.618ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.559378ms iteration = 16 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
132202025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 24 reason = Timeout
132212025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132222025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 40 reason = Timeout
132232025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 26 reason = Timeout
132242025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132252025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
132262025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 46 reason = Timeout
132272025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 91 reason = Timeout
132282025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132292025-11-13T08:06:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
132302025-11-13T08:06:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132312025-11-13T08:06:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132322025-11-13T08:06:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
132332025-11-13T08:06:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
132342025-11-13T08:06:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
132352025-11-13T08:06:32.631ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132362025-11-13T08:06:32.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132372025-11-13T08:06:32.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132382025-11-13T08:06:32.632ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132392025-11-13T08:06:32.633ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132402025-11-13T08:06:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
132412025-11-13T08:06:32.634ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132422025-11-13T08:06:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132432025-11-13T08:06:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
132442025-11-13T08:06:32.637ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132452025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
132462025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132472025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132482025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
132492025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132502025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
132512025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
132522025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132532025-11-13T08:06:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
132542025-11-13T08:06:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
132552025-11-13T08:06:32.639ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
132562025-11-13T08:06:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
132572025-11-13T08:06:32.641ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132582025-11-13T08:06:32.641ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 16 reason = Timeout
132592025-11-13T08:06:32.641ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 24 reason = Timeout
132602025-11-13T08:06:32.641ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
132612025-11-13T08:06:32.641ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132622025-11-13T08:06:32.641ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
132632025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 16 reason = Timeout
132642025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 16 reason = Timeout
132652025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 16 reason = Timeout
132662025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 16 reason = Timeout
132672025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
132682025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
132692025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 16 reason = Timeout
132702025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
132712025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
132722025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
132732025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 46 reason = Timeout
132742025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
132752025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
132762025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4ba91f9a-a9b4-4c47-93bd-d51c608704f2 original_time_started = 2025-11-13 08:06:04.188335 UTC
132772025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 15.553266ms iteration = 40 status = {"Loaded":{"collection_id":"4ba91f9a-a9b4-4c47-93bd-d51c608704f2","time_loaded":"2025-11-13T08:06:32.626673490Z","time_started":"2025-11-13T08:06:04.188335Z"}}
132782025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132792025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
132802025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
132812025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
132822025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
132832025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
132842025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
132852025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
132862025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
132872025-11-13T08:06:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
132882025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
132892025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
132902025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
132912025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132922025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
132932025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 18.948302ms iteration = 91 status = {"config_updated":false}
132942025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132952025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
132962025-11-13T08:06:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
132972025-11-13T08:06:32.647ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
132982025-11-13T08:06:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
132992025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 16 reason = Timeout
133002025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 16 reason = Timeout
133012025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 16 reason = Timeout
133022025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
133032025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
133042025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
133052025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 16 reason = Timeout
133062025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
133072025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 16 reason = Timeout
133082025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
133092025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 31 reason = Timeout
133102025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 16 reason = Timeout
133112025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
133122025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 31 reason = Timeout
133132025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
133142025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
133152025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
133162025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
133172025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
133182025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
133192025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
133202025-11-13T08:06:32.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6401, tv_nsec: 1175862 } })
133212025-11-13T08:06:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
133222025-11-13T08:06:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
133232025-11-13T08:06:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
133242025-11-13T08:06:32.649ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133252025-11-13T08:06:32.650ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133262025-11-13T08:06:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
133272025-11-13T08:06:32.650ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
133282025-11-13T08:06:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
133292025-11-13T08:06:32.650ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.54324ms iteration = 16 status = {"errors":[],"finish_invoked_ok":[]}
133302025-11-13T08:06:32.651ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
133312025-11-13T08:06:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
133322025-11-13T08:06:32.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
133332025-11-13T08:06:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
133342025-11-13T08:06:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
133352025-11-13T08:06:32.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
133362025-11-13T08:06:32.654ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
133372025-11-13T08:06:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
133382025-11-13T08:06:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
133392025-11-13T08:06:32.656ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
133402025-11-13T08:06:32.656ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6401, tv_nsec: 8676860 } })
133412025-11-13T08:06:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
133422025-11-13T08:06:32.656ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
133432025-11-13T08:06:32.668ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 26.831367ms iteration = 16 status = 0
133442025-11-13T08:06:32.705ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
133452025-11-13T08:06:32.705ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 78.695017ms iteration = 46 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
133462025-11-13T08:06:32.714ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
133472025-11-13T08:06:32.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
133482025-11-13T08:06:32.717ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
133492025-11-13T08:06:32.717ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
133502025-11-13T08:06:32.717ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
133512025-11-13T08:06:32.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
133522025-11-13T08:06:32.717ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.678168ms iteration = 46 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
133532025-11-13T08:06:32.717ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133542025-11-13T08:06:32.720ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
133552025-11-13T08:06:32.720ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
133562025-11-13T08:06:32.720ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
133572025-11-13T08:06:32.720ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
133582025-11-13T08:06:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
133592025-11-13T08:06:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
133602025-11-13T08:06:32.721ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
133612025-11-13T08:06:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
133622025-11-13T08:06:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
133632025-11-13T08:06:32.726ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 78.683137ms iteration = 16 status = {"errors":[],"garbage_collect_requested":[]}
133642025-11-13T08:06:32.728ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133652025-11-13T08:06:32.730ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
133662025-11-13T08:06:32.733ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133672025-11-13T08:06:32.733ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
133682025-11-13T08:06:32.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
133692025-11-13T08:06:32.734ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
133702025-11-13T08:06:32.734ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133712025-11-13T08:06:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
133722025-11-13T08:06:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
133732025-11-13T08:06:32.735ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
133742025-11-13T08:06:32.735ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 109.072365ms iteration = 24 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
133752025-11-13T08:06:32.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
133762025-11-13T08:06:32.737ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
133772025-11-13T08:06:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
133782025-11-13T08:06:32.740ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133792025-11-13T08:06:32.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
133802025-11-13T08:06:32.744ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
133812025-11-13T08:06:32.744ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133822025-11-13T08:06:32.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
133832025-11-13T08:06:32.745ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 97.029827ms iteration = 31 status = "NoSitrep"
133842025-11-13T08:06:32.748ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
133852025-11-13T08:06:32.749ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
133862025-11-13T08:06:32.749ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 100.905304ms iteration = 31 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
133872025-11-13T08:06:32.749ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
133882025-11-13T08:06:32.749ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 107.33552ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
133892025-11-13T08:06:32.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
133902025-11-13T08:06:32.751ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133912025-11-13T08:06:32.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
133922025-11-13T08:06:32.753ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133932025-11-13T08:06:32.754ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 106.415292ms iteration = 16 status = {"errors":[],"requests_created_ok":[]}
133942025-11-13T08:06:32.755ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133952025-11-13T08:06:32.758ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
133962025-11-13T08:06:32.758ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
133972025-11-13T08:06:32.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
133982025-11-13T08:06:32.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
133992025-11-13T08:06:32.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134002025-11-13T08:06:32.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134012025-11-13T08:06:32.762ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134022025-11-13T08:06:32.765ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
134032025-11-13T08:06:32.765ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
134042025-11-13T08:06:32.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
134052025-11-13T08:06:32.790ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 141.992842ms iteration = 16 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
134062025-11-13T08:06:32.795ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
134072025-11-13T08:06:32.795ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
134082025-11-13T08:06:32.795ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 153.045588ms iteration = 16 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
134092025-11-13T08:06:32.803ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
134102025-11-13T08:06:32.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
134112025-11-13T08:06:32.805ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 157.253962ms iteration = 16 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
134122025-11-13T08:06:32.805ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
134132025-11-13T08:06:32.805ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
134142025-11-13T08:06:32.805ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9a127e0-427a-4e52-b439-7d7fc134d053", "content-length": "26", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134152025-11-13T08:06:32.805ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
134162025-11-13T08:06:32.807ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
134172025-11-13T08:06:32.809ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 183.256853ms iteration = 26 status = {"physical_disks_added":0}
134182025-11-13T08:06:32.822ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
134192025-11-13T08:06:32.822ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 180.856896ms iteration = 16 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
134202025-11-13T08:06:32.934ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
134212025-11-13T08:06:32.934ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "512c61cd-3c39-4aaa-9749-686d271c3d73", "content-length": "691", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134222025-11-13T08:06:32.934ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
134232025-11-13T08:06:32.934ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
134242025-11-13T08:06:32.934ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
134252025-11-13T08:06:32.936ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 294.845837ms iteration = 16 status = {}
134262025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
134272025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "b64d2454-aa87-4472-8fd6-8d8366b56446", "content-length": "78", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134282025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
134292025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
134302025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
134312025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
134322025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
134332025-11-13T08:06:32.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
134342025-11-13T08:06:32.948ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
134352025-11-13T08:06:32.948ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
134362025-11-13T08:06:32.948ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
134372025-11-13T08:06:32.951ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
134382025-11-13T08:06:32.951ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
134392025-11-13T08:06:32.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
134402025-11-13T08:06:32.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
134412025-11-13T08:06:32.951ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
134422025-11-13T08:06:32.951ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
134432025-11-13T08:06:32.951ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
134442025-11-13T08:06:32.951ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
134452025-11-13T08:06:32.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
134462025-11-13T08:06:32.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
134472025-11-13T08:06:32.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
134482025-11-13T08:06:32.953ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
134492025-11-13T08:06:32.953ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
134502025-11-13T08:06:32.953ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
134512025-11-13T08:06:32.953ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
134522025-11-13T08:06:32.954ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "07611355-8dc4-4a0c-b87a-e88d66a63312", "content-length": "81", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134532025-11-13T08:06:32.954ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "25c911a0-82d1-44e9-be19-d586ee543f3c", "content-length": "81", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134542025-11-13T08:06:32.957ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "da26ee9a-79f8-45c7-b34a-76c3e90e4fa8", "content-length": "372", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134552025-11-13T08:06:32.957ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134562025-11-13T08:06:32.957ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134572025-11-13T08:06:32.958ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
134582025-11-13T08:06:32.959ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 352.813986ms iteration = 16 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
134592025-11-13T08:06:32.962ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
134602025-11-13T08:06:32.962ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
134612025-11-13T08:06:32.962ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d213f798-a168-4078-8a81-00dcc7b5c4bb", "content-length": "26", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134622025-11-13T08:06:32.962ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
134632025-11-13T08:06:32.964ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
134642025-11-13T08:06:32.969ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6401, tv_nsec: 321962729 } })
134652025-11-13T08:06:32.990ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
134662025-11-13T08:06:32.990ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
134672025-11-13T08:06:32.990ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec87bb35-5a9a-49d3-98e0-83ca919714e4", "content-length": "81", "date": "Thu, 13 Nov 2025 08:06:32 GMT"} })
134682025-11-13T08:06:32.992ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 344.445212ms iteration = 16 status = {"disabled":false,"errors":[],"sps":[]}
134692025-11-13T08:06:33.004ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 397.719519ms iteration = 16 status = {}
134702025-11-13T08:06:33.004ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
134712025-11-13T08:06:33.004ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
134722025-11-13T08:06:33.005ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f0ef9ecf-58ba-4948-88b1-f1a06304ee26", "date": "Thu, 13 Nov 2025 08:06:33 GMT"} })
134732025-11-13T08:06:33.005ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134742025-11-13T08:06:33.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134752025-11-13T08:06:33.008ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134762025-11-13T08:06:33.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134772025-11-13T08:06:33.011ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
134782025-11-13T08:06:33.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
134792025-11-13T08:06:33.015ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
134802025-11-13T08:06:33.015ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
134812025-11-13T08:06:33.015ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "553ce987-3243-49f4-84e7-f3904a5b3942", "date": "Thu, 13 Nov 2025 08:06:33 GMT"} })
134822025-11-13T08:06:33.017ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 375.628415ms iteration = 24 status = {}
134832025-11-13T08:06:33.036ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134842025-11-13T08:06:33.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134852025-11-13T08:06:33.038ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
134862025-11-13T08:06:33.038ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
134872025-11-13T08:06:33.039ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
134882025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
134892025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
134902025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
134912025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
134922025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134932025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134942025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134952025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
134962025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
134972025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
134982025-11-13T08:06:33.050ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
134992025-11-13T08:06:33.050ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135002025-11-13T08:06:33.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135012025-11-13T08:06:33.053ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
135022025-11-13T08:06:33.053ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
135032025-11-13T08:06:33.228ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f2d7e525-712d-4441-9aa4-ef81bb2445ed", "date": "Thu, 13 Nov 2025 08:06:33 GMT"} })
135042025-11-13T08:06:33.228ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
135052025-11-13T08:06:33.228ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
135062025-11-13T08:06:33.228ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d353c53d-ec61-442c-a515-5d6635e502cb", "content-length": "682", "date": "Thu, 13 Nov 2025 08:06:33 GMT"} })
135072025-11-13T08:06:33.228ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135082025-11-13T08:06:33.229ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135092025-11-13T08:06:33.232ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135102025-11-13T08:06:33.232ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135112025-11-13T08:06:33.236ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135122025-11-13T08:06:33.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135132025-11-13T08:06:33.238ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
135142025-11-13T08:06:33.238ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
135152025-11-13T08:06:33.244ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 602.847242ms iteration = 16 status = {}
135162025-11-13T08:06:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 321 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 206e2670-bd88-42fc-b0e1-6d16b0894e02 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
135172025-11-13T08:06:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 92 reason = Timeout
135182025-11-13T08:06:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135192025-11-13T08:06:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135202025-11-13T08:06:37.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135212025-11-13T08:06:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
135222025-11-13T08:06:37.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135232025-11-13T08:06:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135242025-11-13T08:06:37.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
135252025-11-13T08:06:37.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.657991ms iteration = 92 status = {"config_updated":false}
135262025-11-13T08:06:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 47 reason = Timeout
135272025-11-13T08:06:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 93 reason = Timeout
135282025-11-13T08:06:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135292025-11-13T08:06:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135302025-11-13T08:06:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135312025-11-13T08:06:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135322025-11-13T08:06:42.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135332025-11-13T08:06:42.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135342025-11-13T08:06:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
135352025-11-13T08:06:42.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135362025-11-13T08:06:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135372025-11-13T08:06:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
135382025-11-13T08:06:42.633ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135392025-11-13T08:06:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135402025-11-13T08:06:42.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135412025-11-13T08:06:42.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135422025-11-13T08:06:42.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135432025-11-13T08:06:42.638ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
135442025-11-13T08:06:42.638ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.755371ms iteration = 93 status = {"config_updated":false}
135452025-11-13T08:06:42.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
135462025-11-13T08:06:42.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
135472025-11-13T08:06:42.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
135482025-11-13T08:06:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 47 reason = Timeout
135492025-11-13T08:06:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135502025-11-13T08:06:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135512025-11-13T08:06:42.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
135522025-11-13T08:06:42.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
135532025-11-13T08:06:42.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.54912ms iteration = 47 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
135542025-11-13T08:06:42.653ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
135552025-11-13T08:06:42.653ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.184894ms iteration = 47 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
135562025-11-13T08:06:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 280 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 129381d0-3fb0-4b96-8024-a21d9bd7e55c response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
135572025-11-13T08:06:47.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 41 reason = Timeout
135582025-11-13T08:06:47.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 94 reason = Timeout
135592025-11-13T08:06:47.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
135602025-11-13T08:06:47.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135612025-11-13T08:06:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
135622025-11-13T08:06:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135632025-11-13T08:06:47.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135642025-11-13T08:06:47.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135652025-11-13T08:06:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
135662025-11-13T08:06:47.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
135672025-11-13T08:06:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
135682025-11-13T08:06:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
135692025-11-13T08:06:47.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
135702025-11-13T08:06:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
135712025-11-13T08:06:47.633ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4ba91f9a-a9b4-4c47-93bd-d51c608704f2 original_time_started = 2025-11-13 08:06:04.188335 UTC
135722025-11-13T08:06:47.633ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.294373ms iteration = 41 status = {"Loaded":{"collection_id":"4ba91f9a-a9b4-4c47-93bd-d51c608704f2","time_loaded":"2025-11-13T08:06:47.626131680Z","time_started":"2025-11-13T08:06:04.188335Z"}}
135732025-11-13T08:06:47.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
135742025-11-13T08:06:47.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.673411ms iteration = 94 status = {"config_updated":false}
135752025-11-13T08:06:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 32 reason = Timeout
135762025-11-13T08:06:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
135772025-11-13T08:06:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 32 reason = Timeout
135782025-11-13T08:06:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
135792025-11-13T08:06:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
135802025-11-13T08:06:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
135812025-11-13T08:06:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
135822025-11-13T08:06:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
135832025-11-13T08:06:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135842025-11-13T08:06:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135852025-11-13T08:06:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
135862025-11-13T08:06:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
135872025-11-13T08:06:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
135882025-11-13T08:06:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
135892025-11-13T08:06:47.653ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 4.857116ms iteration = 32 status = "NoSitrep"
135902025-11-13T08:06:47.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
135912025-11-13T08:06:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
135922025-11-13T08:06:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
135932025-11-13T08:06:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
135942025-11-13T08:06:47.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135952025-11-13T08:06:47.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
135962025-11-13T08:06:47.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
135972025-11-13T08:06:47.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.328203ms iteration = 32 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
135982025-11-13T08:06:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 48 reason = Timeout
135992025-11-13T08:06:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 95 reason = Timeout
136002025-11-13T08:06:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136012025-11-13T08:06:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
136022025-11-13T08:06:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
136032025-11-13T08:06:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
136042025-11-13T08:06:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136052025-11-13T08:06:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136062025-11-13T08:06:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
136072025-11-13T08:06:52.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
136082025-11-13T08:06:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
136092025-11-13T08:06:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
136102025-11-13T08:06:52.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136112025-11-13T08:06:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
136122025-11-13T08:06:52.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
136132025-11-13T08:06:52.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.337905ms iteration = 95 status = {"config_updated":false}
136142025-11-13T08:06:52.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136152025-11-13T08:06:52.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
136162025-11-13T08:06:52.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136172025-11-13T08:06:52.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
136182025-11-13T08:06:52.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
136192025-11-13T08:06:52.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
136202025-11-13T08:06:52.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 48 reason = Timeout
136212025-11-13T08:06:52.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
136222025-11-13T08:06:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
136232025-11-13T08:06:52.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
136242025-11-13T08:06:52.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
136252025-11-13T08:06:52.649ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
136262025-11-13T08:06:52.649ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.557663ms iteration = 48 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
136272025-11-13T08:06:52.683ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.258101ms iteration = 48 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
136282025-11-13T08:06:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 450 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 6848e7ce-a1e1-4bed-a04c-c9ff217e9979 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
136292025-11-13T08:06:57.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 96 reason = Timeout
136302025-11-13T08:06:57.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
136312025-11-13T08:06:57.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
136322025-11-13T08:06:57.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136332025-11-13T08:06:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
136342025-11-13T08:06:57.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
136352025-11-13T08:06:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
136362025-11-13T08:06:57.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
136372025-11-13T08:06:57.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.582479ms iteration = 96 status = {"config_updated":false}
136382025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 9 reason = Timeout
136392025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 9 reason = Timeout
136402025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 17 reason = Timeout
136412025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136422025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 9 reason = Timeout
136432025-11-13T08:07:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:57:02.607597081 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
136442025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
136452025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 9 reason = Timeout
136462025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
136472025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136482025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 9 reason = Timeout
136492025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 17 reason = Timeout
136502025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
136512025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 11 reason = Timeout
136522025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 17 reason = Timeout
136532025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 11 reason = Timeout
136542025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
136552025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 9 reason = Timeout
136562025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
136572025-11-13T08:07:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
136582025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
136592025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 10 reason = Timeout
136602025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136612025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
136622025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
136632025-11-13T08:07:02.608ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
136642025-11-13T08:07:02.608ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = ExternalNetworking step id = Ensure
136652025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
136662025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
136672025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136682025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
136692025-11-13T08:07:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6430, tv_nsec: 960892847 } })
136702025-11-13T08:07:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6430, tv_nsec: 960911558 } })
136712025-11-13T08:07:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136722025-11-13T08:07:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
136732025-11-13T08:07:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
136742025-11-13T08:07:02.609ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
136752025-11-13T08:07:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 2.171143ms iteration = 9 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
136762025-11-13T08:07:02.609ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
136772025-11-13T08:07:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 2.245834ms iteration = 9 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
136782025-11-13T08:07:02.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
136792025-11-13T08:07:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
136802025-11-13T08:07:02.611ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6430, tv_nsec: 963847315 } })
136812025-11-13T08:07:02.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6430, tv_nsec: 963960898 } })
136822025-11-13T08:07:02.611ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136832025-11-13T08:07:02.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136842025-11-13T08:07:02.612ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
136852025-11-13T08:07:02.613ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:57:02.607597081 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
136862025-11-13T08:07:02.613ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.983288ms iteration = 9 status = {"expiration":"2025-11-13T07:57:02.607597081Z","pruned":[]}
136872025-11-13T08:07:02.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136882025-11-13T08:07:02.615ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
136892025-11-13T08:07:02.615ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
136902025-11-13T08:07:02.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
136912025-11-13T08:07:02.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
136922025-11-13T08:07:02.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
136932025-11-13T08:07:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 9 reason = Timeout
136942025-11-13T08:07:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
136952025-11-13T08:07:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 21 reason = Timeout
136962025-11-13T08:07:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
136972025-11-13T08:07:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
136982025-11-13T08:07:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
136992025-11-13T08:07:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
137002025-11-13T08:07:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
137012025-11-13T08:07:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
137022025-11-13T08:07:02.622ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
137032025-11-13T08:07:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
137042025-11-13T08:07:02.624ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
137052025-11-13T08:07:02.624ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
137062025-11-13T08:07:02.624ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
137072025-11-13T08:07:02.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
137082025-11-13T08:07:02.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
137092025-11-13T08:07:02.624ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 17.005974ms iteration = 17 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
137102025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137112025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 42 reason = Timeout
137122025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 25 reason = Timeout
137132025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
137142025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
137152025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 49 reason = Timeout
137162025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 27 reason = Timeout
137172025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
137182025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
137192025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 97 reason = Timeout
137202025-11-13T08:07:02.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
137212025-11-13T08:07:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
137222025-11-13T08:07:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
137232025-11-13T08:07:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
137242025-11-13T08:07:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
137252025-11-13T08:07:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
137262025-11-13T08:07:02.626ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137272025-11-13T08:07:02.627ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
137282025-11-13T08:07:02.627ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
137292025-11-13T08:07:02.627ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
137302025-11-13T08:07:02.627ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
137312025-11-13T08:07:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
137322025-11-13T08:07:02.628ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137332025-11-13T08:07:02.629ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137342025-11-13T08:07:02.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137352025-11-13T08:07:02.630ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137362025-11-13T08:07:02.630ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137372025-11-13T08:07:02.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137382025-11-13T08:07:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
137392025-11-13T08:07:02.632ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
137402025-11-13T08:07:02.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
137412025-11-13T08:07:02.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
137422025-11-13T08:07:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
137432025-11-13T08:07:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
137442025-11-13T08:07:02.634ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
137452025-11-13T08:07:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
137462025-11-13T08:07:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
137472025-11-13T08:07:02.636ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
137482025-11-13T08:07:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
137492025-11-13T08:07:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
137502025-11-13T08:07:02.636ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
137512025-11-13T08:07:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
137522025-11-13T08:07:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
137532025-11-13T08:07:02.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
137542025-11-13T08:07:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
137552025-11-13T08:07:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
137562025-11-13T08:07:02.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
137572025-11-13T08:07:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
137582025-11-13T08:07:02.638ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
137592025-11-13T08:07:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
137602025-11-13T08:07:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
137612025-11-13T08:07:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
137622025-11-13T08:07:02.639ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
137632025-11-13T08:07:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
137642025-11-13T08:07:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 9 reason = Timeout
137652025-11-13T08:07:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
137662025-11-13T08:07:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
137672025-11-13T08:07:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 9 reason = Timeout
137682025-11-13T08:07:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
137692025-11-13T08:07:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
137702025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 17 reason = Timeout
137712025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
137722025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
137732025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 17 reason = Timeout
137742025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
137752025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 17 reason = Timeout
137762025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
137772025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
137782025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 17 reason = Timeout
137792025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 49 reason = Timeout
137802025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
137812025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
137822025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 25 reason = Timeout
137832025-11-13T08:07:02.643ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
137842025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
137852025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 17 reason = Timeout
137862025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
137872025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 17 reason = Timeout
137882025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
137892025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
137902025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
137912025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
137922025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
137932025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
137942025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
137952025-11-13T08:07:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
137962025-11-13T08:07:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
137972025-11-13T08:07:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
137982025-11-13T08:07:02.646ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 9 reason = Timeout
137992025-11-13T08:07:02.646ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
138002025-11-13T08:07:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
138012025-11-13T08:07:02.647ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
138022025-11-13T08:07:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
138032025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 9 reason = Timeout
138042025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
138052025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 17 reason = Timeout
138062025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
138072025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 17 reason = Timeout
138082025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 33 reason = Timeout
138092025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
138102025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 17 reason = Timeout
138112025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
138122025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 17 reason = Timeout
138132025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
138142025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 9 reason = Timeout
138152025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
138162025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 17 reason = Timeout
138172025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
138182025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138192025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 9 reason = Timeout
138202025-11-13T08:07:02.651ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
138212025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
138222025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 17 reason = Timeout
138232025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 33 reason = Timeout
138242025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
138252025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
138262025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
138272025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
138282025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
138292025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
138302025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
138312025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
138322025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
138332025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
138342025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
138352025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 25.855779ms iteration = 97 status = {"config_updated":false}
138362025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
138372025-11-13T08:07:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
138382025-11-13T08:07:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
138392025-11-13T08:07:02.652ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6431, tv_nsec: 4273210 } })
138402025-11-13T08:07:02.652ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138412025-11-13T08:07:02.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138422025-11-13T08:07:02.656ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138432025-11-13T08:07:02.657ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4ba91f9a-a9b4-4c47-93bd-d51c608704f2 original_time_started = 2025-11-13 08:06:04.188335 UTC
138442025-11-13T08:07:02.657ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 31.685893ms iteration = 42 status = {"Loaded":{"collection_id":"4ba91f9a-a9b4-4c47-93bd-d51c608704f2","time_loaded":"2025-11-13T08:07:02.625898781Z","time_started":"2025-11-13T08:06:04.188335Z"}}
138452025-11-13T08:07:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
138462025-11-13T08:07:02.658ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
138472025-11-13T08:07:02.658ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
138482025-11-13T08:07:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
138492025-11-13T08:07:02.659ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 12.872533ms iteration = 9 status = {"errors":[],"found_port_ok":[]}
138502025-11-13T08:07:02.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
138512025-11-13T08:07:02.660ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
138522025-11-13T08:07:02.660ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
138532025-11-13T08:07:02.660ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
138542025-11-13T08:07:02.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
138552025-11-13T08:07:02.663ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
138562025-11-13T08:07:02.667ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
138572025-11-13T08:07:02.675ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
138582025-11-13T08:07:02.675ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 24.130024ms iteration = 9 status = {"by_rx":{},"error":null}
138592025-11-13T08:07:02.683ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
138602025-11-13T08:07:02.687ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
138612025-11-13T08:07:02.699ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
138622025-11-13T08:07:02.711ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
138632025-11-13T08:07:02.713ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
138642025-11-13T08:07:02.717ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
138652025-11-13T08:07:02.718ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
138662025-11-13T08:07:02.728ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
138672025-11-13T08:07:02.730ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
138682025-11-13T08:07:02.735ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
138692025-11-13T08:07:02.742ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = SledList step id = Fetch
138702025-11-13T08:07:02.742ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
138712025-11-13T08:07:02.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
138722025-11-13T08:07:02.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138732025-11-13T08:07:02.748ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138742025-11-13T08:07:02.749ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138752025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
138762025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
138772025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
138782025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
138792025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
138802025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
138812025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
138822025-11-13T08:07:02.753ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
138832025-11-13T08:07:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
138842025-11-13T08:07:02.754ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
138852025-11-13T08:07:02.755ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
138862025-11-13T08:07:02.755ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
138872025-11-13T08:07:02.755ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
138882025-11-13T08:07:02.758ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138892025-11-13T08:07:02.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
138902025-11-13T08:07:02.762ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
138912025-11-13T08:07:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
138922025-11-13T08:07:02.770ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
138932025-11-13T08:07:02.771ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
138942025-11-13T08:07:02.771ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
138952025-11-13T08:07:02.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
138962025-11-13T08:07:02.777ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138972025-11-13T08:07:02.779ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
138982025-11-13T08:07:02.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138992025-11-13T08:07:02.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
139002025-11-13T08:07:02.781ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
139012025-11-13T08:07:02.781ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
139022025-11-13T08:07:02.781ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 162.713219ms iteration = 9 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
139032025-11-13T08:07:02.781ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
139042025-11-13T08:07:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
139052025-11-13T08:07:02.781ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
139062025-11-13T08:07:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
139072025-11-13T08:07:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
139082025-11-13T08:07:02.783ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139092025-11-13T08:07:02.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139102025-11-13T08:07:02.784ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
139112025-11-13T08:07:02.784ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
139122025-11-13T08:07:02.784ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
139132025-11-13T08:07:02.784ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
139142025-11-13T08:07:02.784ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
139152025-11-13T08:07:02.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139162025-11-13T08:07:02.784ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 133.543265ms iteration = 17 status = {"errors":[],"finish_invoked_ok":[]}
139172025-11-13T08:07:02.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
139182025-11-13T08:07:02.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
139192025-11-13T08:07:02.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139202025-11-13T08:07:02.786ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
139212025-11-13T08:07:02.787ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 136.257589ms iteration = 33 status = "NoSitrep"
139222025-11-13T08:07:02.787ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 144.627544ms iteration = 17 status = 0
139232025-11-13T08:07:02.788ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
139242025-11-13T08:07:02.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139252025-11-13T08:07:02.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139262025-11-13T08:07:02.790ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139272025-11-13T08:07:02.791ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139282025-11-13T08:07:02.792ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
139292025-11-13T08:07:02.793ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
139302025-11-13T08:07:02.793ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
139312025-11-13T08:07:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
139322025-11-13T08:07:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
139332025-11-13T08:07:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
139342025-11-13T08:07:02.794ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6431, tv_nsec: 146821663 } })
139352025-11-13T08:07:02.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139362025-11-13T08:07:02.795ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
139372025-11-13T08:07:02.795ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139382025-11-13T08:07:02.795ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139392025-11-13T08:07:02.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
139402025-11-13T08:07:02.796ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
139412025-11-13T08:07:02.796ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139422025-11-13T08:07:02.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
139432025-11-13T08:07:02.796ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
139442025-11-13T08:07:02.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
139452025-11-13T08:07:02.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139462025-11-13T08:07:02.797ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
139472025-11-13T08:07:02.797ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
139482025-11-13T08:07:02.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
139492025-11-13T08:07:02.798ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139502025-11-13T08:07:02.799ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139512025-11-13T08:07:02.800ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
139522025-11-13T08:07:02.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139532025-11-13T08:07:02.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139542025-11-13T08:07:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
139552025-11-13T08:07:02.802ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
139562025-11-13T08:07:02.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
139572025-11-13T08:07:02.803ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
139582025-11-13T08:07:02.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
139592025-11-13T08:07:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
139602025-11-13T08:07:02.803ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
139612025-11-13T08:07:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
139622025-11-13T08:07:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
139632025-11-13T08:07:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
139642025-11-13T08:07:02.804ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
139652025-11-13T08:07:02.804ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
139662025-11-13T08:07:02.804ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
139672025-11-13T08:07:02.804ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
139682025-11-13T08:07:02.804ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
139692025-11-13T08:07:02.804ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 153.401866ms iteration = 33 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
139702025-11-13T08:07:02.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
139712025-11-13T08:07:02.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
139722025-11-13T08:07:02.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139732025-11-13T08:07:02.807ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139742025-11-13T08:07:02.808ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139752025-11-13T08:07:02.814ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
139762025-11-13T08:07:02.814ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
139772025-11-13T08:07:02.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
139782025-11-13T08:07:02.815ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
139792025-11-13T08:07:02.815ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
139802025-11-13T08:07:02.815ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
139812025-11-13T08:07:02.815ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 207.776295ms iteration = 9 status = {"generation":1}
139822025-11-13T08:07:02.819ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 168.033794ms iteration = 9 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
139832025-11-13T08:07:02.821ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
139842025-11-13T08:07:02.821ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
139852025-11-13T08:07:02.821ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
139862025-11-13T08:07:02.821ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 214.005397ms iteration = 9 status = {"generation":2}
139872025-11-13T08:07:02.826ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 175.682154ms iteration = 17 status = {"errors":[],"requests_created_ok":[]}
139882025-11-13T08:07:02.827ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139892025-11-13T08:07:02.831ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
139902025-11-13T08:07:02.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139912025-11-13T08:07:02.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139922025-11-13T08:07:02.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139932025-11-13T08:07:02.837ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
139942025-11-13T08:07:02.837ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
139952025-11-13T08:07:02.837ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 194.883322ms iteration = 9 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
139962025-11-13T08:07:02.839ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
139972025-11-13T08:07:02.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
139982025-11-13T08:07:02.839ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
139992025-11-13T08:07:02.860ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
140002025-11-13T08:07:02.860ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 234.659434ms iteration = 49 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
140012025-11-13T08:07:02.883ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
140022025-11-13T08:07:02.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
140032025-11-13T08:07:02.890ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
140042025-11-13T08:07:02.890ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 282.528935ms iteration = 9 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
140052025-11-13T08:07:02.893ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
140062025-11-13T08:07:02.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
140072025-11-13T08:07:02.904ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
140082025-11-13T08:07:02.904ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 278.311541ms iteration = 25 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
140092025-11-13T08:07:02.910ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
140102025-11-13T08:07:02.910ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
140112025-11-13T08:07:02.910ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
140122025-11-13T08:07:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
140132025-11-13T08:07:02.917ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 266.030199ms iteration = 17 status = {"errors":[],"garbage_collect_requested":[]}
140142025-11-13T08:07:02.917ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 273.839044ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
140152025-11-13T08:07:02.918ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 267.382046ms iteration = 17 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
140162025-11-13T08:07:02.923ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140172025-11-13T08:07:02.926ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
140182025-11-13T08:07:02.926ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
140192025-11-13T08:07:02.927ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b230418c-22a6-4ebd-b8db-5e1cdc40d0be", "content-length": "26", "date": "Thu, 13 Nov 2025 08:07:02 GMT"} })
140202025-11-13T08:07:02.927ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
140212025-11-13T08:07:02.927ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
140222025-11-13T08:07:02.927ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
140232025-11-13T08:07:02.928ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
140242025-11-13T08:07:02.929ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
140252025-11-13T08:07:02.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140262025-11-13T08:07:02.953ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140272025-11-13T08:07:02.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140282025-11-13T08:07:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140292025-11-13T08:07:02.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140302025-11-13T08:07:02.961ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
140312025-11-13T08:07:02.961ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 317.981411ms iteration = 17 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
140322025-11-13T08:07:02.962ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 336.162979ms iteration = 27 status = {"physical_disks_added":0}
140332025-11-13T08:07:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
140342025-11-13T08:07:02.966ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140352025-11-13T08:07:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140362025-11-13T08:07:02.969ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
140372025-11-13T08:07:02.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
140382025-11-13T08:07:02.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140392025-11-13T08:07:02.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140402025-11-13T08:07:02.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140412025-11-13T08:07:02.985ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
140422025-11-13T08:07:02.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140432025-11-13T08:07:02.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140442025-11-13T08:07:02.987ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
140452025-11-13T08:07:02.987ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 344.286549ms iteration = 49 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
140462025-11-13T08:07:02.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140472025-11-13T08:07:02.988ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
140482025-11-13T08:07:02.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
140492025-11-13T08:07:02.992ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
140502025-11-13T08:07:02.992ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
140512025-11-13T08:07:02.992ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 349.374168ms iteration = 17 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
140522025-11-13T08:07:02.993ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
140532025-11-13T08:07:02.993ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140542025-11-13T08:07:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140552025-11-13T08:07:02.997ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140562025-11-13T08:07:02.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140572025-11-13T08:07:02.998ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 390.311643ms iteration = 17 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
140582025-11-13T08:07:02.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140592025-11-13T08:07:03.004ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
140602025-11-13T08:07:03.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140612025-11-13T08:07:03.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140622025-11-13T08:07:03.006ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
140632025-11-13T08:07:03.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
140642025-11-13T08:07:03.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140652025-11-13T08:07:03.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140662025-11-13T08:07:03.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140672025-11-13T08:07:03.015ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = DeployNexusRecords step id = Ensure
140682025-11-13T08:07:03.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140692025-11-13T08:07:03.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140702025-11-13T08:07:03.017ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
140712025-11-13T08:07:03.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140722025-11-13T08:07:03.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140732025-11-13T08:07:03.018ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140742025-11-13T08:07:03.020ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
140752025-11-13T08:07:03.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140762025-11-13T08:07:03.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140772025-11-13T08:07:03.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140782025-11-13T08:07:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140792025-11-13T08:07:03.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140802025-11-13T08:07:03.024ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
140812025-11-13T08:07:03.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
140822025-11-13T08:07:03.028ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140832025-11-13T08:07:03.030ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
140842025-11-13T08:07:03.030ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
140852025-11-13T08:07:03.031ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
140862025-11-13T08:07:03.033ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
140872025-11-13T08:07:03.043ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
140882025-11-13T08:07:03.043ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "528fbd9f-b4b4-43f3-b04f-005b936116db", "content-length": "78", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
140892025-11-13T08:07:03.044ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
140902025-11-13T08:07:03.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
140912025-11-13T08:07:03.044ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
140922025-11-13T08:07:03.045ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
140932025-11-13T08:07:03.045ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
140942025-11-13T08:07:03.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
140952025-11-13T08:07:03.048ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
140962025-11-13T08:07:03.049ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
140972025-11-13T08:07:03.053ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
140982025-11-13T08:07:03.053ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
140992025-11-13T08:07:03.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
141002025-11-13T08:07:03.054ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
141012025-11-13T08:07:03.054ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
141022025-11-13T08:07:03.055ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
141032025-11-13T08:07:03.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141042025-11-13T08:07:03.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141052025-11-13T08:07:03.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141062025-11-13T08:07:03.063ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141072025-11-13T08:07:03.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141082025-11-13T08:07:03.068ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 417.527188ms iteration = 17 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
141092025-11-13T08:07:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
141102025-11-13T08:07:03.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141112025-11-13T08:07:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141122025-11-13T08:07:03.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141132025-11-13T08:07:03.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
141142025-11-13T08:07:03.082ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141152025-11-13T08:07:03.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141162025-11-13T08:07:03.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141172025-11-13T08:07:03.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141182025-11-13T08:07:03.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141192025-11-13T08:07:03.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141202025-11-13T08:07:03.091ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
141212025-11-13T08:07:03.094ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141222025-11-13T08:07:03.098ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
141232025-11-13T08:07:03.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
141242025-11-13T08:07:03.100ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
141252025-11-13T08:07:03.100ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141262025-11-13T08:07:03.100ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141272025-11-13T08:07:03.105ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141282025-11-13T08:07:03.109ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
141292025-11-13T08:07:03.109ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
141302025-11-13T08:07:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
141312025-11-13T08:07:03.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141322025-11-13T08:07:03.114ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
141332025-11-13T08:07:03.114ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141342025-11-13T08:07:03.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141352025-11-13T08:07:03.118ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141362025-11-13T08:07:03.118ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6431, tv_nsec: 471038817 } })
141372025-11-13T08:07:03.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141382025-11-13T08:07:03.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141392025-11-13T08:07:03.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141402025-11-13T08:07:03.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141412025-11-13T08:07:03.123ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = SledAgent step id = Ensure
141422025-11-13T08:07:03.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
141432025-11-13T08:07:03.129ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
141442025-11-13T08:07:03.129ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
141452025-11-13T08:07:03.130ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aa1316b-798b-4103-aabb-36307c29b4a8", "content-length": "81", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
141462025-11-13T08:07:03.131ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
141472025-11-13T08:07:03.131ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
141482025-11-13T08:07:03.132ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
141492025-11-13T08:07:03.132ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "afe9573b-e38a-4038-8f5a-228a3d512fd5", "content-length": "81", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
141502025-11-13T08:07:03.135ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
141512025-11-13T08:07:03.135ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
141522025-11-13T08:07:03.135ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
141532025-11-13T08:07:03.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
141542025-11-13T08:07:03.158ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
141552025-11-13T08:07:03.158ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
141562025-11-13T08:07:03.159ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b9532f3-d1e5-407d-b1c5-9fa44546af53", "content-length": "26", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
141572025-11-13T08:07:03.159ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
141582025-11-13T08:07:03.162ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
141592025-11-13T08:07:03.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141602025-11-13T08:07:03.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141612025-11-13T08:07:03.207ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
141622025-11-13T08:07:03.207ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
141632025-11-13T08:07:03.207ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "94f7a16b-fb63-4966-bc43-145be9baae14", "content-length": "372", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
141642025-11-13T08:07:03.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
141652025-11-13T08:07:03.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141662025-11-13T08:07:03.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
141672025-11-13T08:07:03.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141682025-11-13T08:07:03.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141692025-11-13T08:07:03.212ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
141702025-11-13T08:07:03.212ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
141712025-11-13T08:07:03.213ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3824b827-5f84-49f1-bfce-d6e1bfd9a484", "content-length": "81", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
141722025-11-13T08:07:03.216ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
141732025-11-13T08:07:03.218ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 567.210271ms iteration = 17 status = {"disabled":false,"errors":[],"sps":[]}
141742025-11-13T08:07:03.247ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
141752025-11-13T08:07:03.247ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
141762025-11-13T08:07:03.249ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141772025-11-13T08:07:03.253ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
141782025-11-13T08:07:03.253ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
141792025-11-13T08:07:03.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
141802025-11-13T08:07:03.255ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141812025-11-13T08:07:03.258ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
141822025-11-13T08:07:03.258ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
141832025-11-13T08:07:03.258ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
141842025-11-13T08:07:03.271ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
141852025-11-13T08:07:03.271ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 628.867213ms iteration = 9 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
141862025-11-13T08:07:03.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141872025-11-13T08:07:03.282ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141882025-11-13T08:07:03.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141892025-11-13T08:07:03.284ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141902025-11-13T08:07:03.287ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141912025-11-13T08:07:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
141922025-11-13T08:07:03.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141932025-11-13T08:07:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141942025-11-13T08:07:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
141952025-11-13T08:07:03.295ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
141962025-11-13T08:07:03.295ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
141972025-11-13T08:07:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
141982025-11-13T08:07:03.295ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "10c1f7b3-1196-499d-b776-4c1f31385386", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
141992025-11-13T08:07:03.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142002025-11-13T08:07:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142012025-11-13T08:07:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142022025-11-13T08:07:03.299ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142032025-11-13T08:07:03.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142042025-11-13T08:07:03.301ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
142052025-11-13T08:07:03.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142062025-11-13T08:07:03.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142072025-11-13T08:07:03.303ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142082025-11-13T08:07:03.303ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142092025-11-13T08:07:03.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142102025-11-13T08:07:03.303ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142112025-11-13T08:07:03.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142122025-11-13T08:07:03.308ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
142132025-11-13T08:07:03.308ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
142142025-11-13T08:07:03.309ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4a673aa7-b4bb-4fe6-8f2f-10146d9ecf1e", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
142152025-11-13T08:07:03.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
142162025-11-13T08:07:03.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142172025-11-13T08:07:03.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142182025-11-13T08:07:03.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142192025-11-13T08:07:03.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142202025-11-13T08:07:03.313ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 670.480671ms iteration = 25 status = {}
142212025-11-13T08:07:03.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142222025-11-13T08:07:03.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
142232025-11-13T08:07:03.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142242025-11-13T08:07:03.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142252025-11-13T08:07:03.361ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
142262025-11-13T08:07:03.361ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
142272025-11-13T08:07:03.361ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 1d44b9b7-6ef5-4430-99cc-c93b68d58f6a parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
142282025-11-13T08:07:03.361ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142292025-11-13T08:07:03.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142302025-11-13T08:07:03.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142312025-11-13T08:07:03.368ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
142322025-11-13T08:07:03.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
142332025-11-13T08:07:03.368ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
142342025-11-13T08:07:03.371ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
142352025-11-13T08:07:03.371ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
142362025-11-13T08:07:03.371ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 752.763579ms iteration = 21 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
142372025-11-13T08:07:03.389ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
142382025-11-13T08:07:03.390ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
142392025-11-13T08:07:03.395ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "0de3c29b-a60e-4bf2-af34-9555e6b0e424", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
142402025-11-13T08:07:03.395ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
142412025-11-13T08:07:03.397ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
142422025-11-13T08:07:03.397ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 746.651438ms iteration = 9 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
142432025-11-13T08:07:03.397ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 26 reason = Dependency
142442025-11-13T08:07:03.397ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
142452025-11-13T08:07:03.397ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142462025-11-13T08:07:03.397ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "79fde4cf-a65e-4c10-b471-6b94a726d9bd", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
142472025-11-13T08:07:03.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142482025-11-13T08:07:03.399ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = FirewallRules step id = Ensure
142492025-11-13T08:07:03.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142502025-11-13T08:07:03.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142512025-11-13T08:07:03.400ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142522025-11-13T08:07:03.402ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
142532025-11-13T08:07:03.402ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142542025-11-13T08:07:03.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142552025-11-13T08:07:03.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142562025-11-13T08:07:03.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142572025-11-13T08:07:03.403ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142582025-11-13T08:07:03.404ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142592025-11-13T08:07:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142602025-11-13T08:07:03.406ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
142612025-11-13T08:07:03.406ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
142622025-11-13T08:07:03.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
142632025-11-13T08:07:03.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142642025-11-13T08:07:03.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142652025-11-13T08:07:03.409ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142662025-11-13T08:07:03.409ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142672025-11-13T08:07:03.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142682025-11-13T08:07:03.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142692025-11-13T08:07:03.411ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 803.794872ms iteration = 17 status = {}
142702025-11-13T08:07:03.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142712025-11-13T08:07:03.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
142722025-11-13T08:07:03.419ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142732025-11-13T08:07:03.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142742025-11-13T08:07:03.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142752025-11-13T08:07:03.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142762025-11-13T08:07:03.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142772025-11-13T08:07:03.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142782025-11-13T08:07:03.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142792025-11-13T08:07:03.432ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
142802025-11-13T08:07:03.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142812025-11-13T08:07:03.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142822025-11-13T08:07:03.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142832025-11-13T08:07:03.439ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142842025-11-13T08:07:03.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142852025-11-13T08:07:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142862025-11-13T08:07:03.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142872025-11-13T08:07:03.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142882025-11-13T08:07:03.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142892025-11-13T08:07:03.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142902025-11-13T08:07:03.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142912025-11-13T08:07:03.453ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142922025-11-13T08:07:03.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
142932025-11-13T08:07:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
142942025-11-13T08:07:03.456ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142952025-11-13T08:07:03.461ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
142962025-11-13T08:07:03.461ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "e22a83a1-135f-4a42-b193-f55a2f6b99ae", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
142972025-11-13T08:07:03.461ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
142982025-11-13T08:07:03.463ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
142992025-11-13T08:07:03.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143002025-11-13T08:07:03.463ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143012025-11-13T08:07:03.463ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "2777669e-f75b-4c83-8b57-2a0ae031dc5e", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
143022025-11-13T08:07:03.463ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
143032025-11-13T08:07:03.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143042025-11-13T08:07:03.465ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143052025-11-13T08:07:03.465ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
143062025-11-13T08:07:03.465ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 858.100099ms iteration = 11 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
143072025-11-13T08:07:03.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143082025-11-13T08:07:03.466ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143092025-11-13T08:07:03.467ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143102025-11-13T08:07:03.467ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143112025-11-13T08:07:03.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143122025-11-13T08:07:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143132025-11-13T08:07:03.470ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
143142025-11-13T08:07:03.470ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143152025-11-13T08:07:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143162025-11-13T08:07:03.472ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143172025-11-13T08:07:03.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143182025-11-13T08:07:03.474ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143192025-11-13T08:07:03.474ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143202025-11-13T08:07:03.475ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143212025-11-13T08:07:03.476ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143222025-11-13T08:07:03.477ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143232025-11-13T08:07:03.483ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
143242025-11-13T08:07:03.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143252025-11-13T08:07:03.483ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143262025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
143272025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
143282025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
143292025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
143302025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143312025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143322025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143332025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
143342025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
143352025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
143362025-11-13T08:07:03.484ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
143372025-11-13T08:07:03.484ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
143382025-11-13T08:07:03.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143392025-11-13T08:07:03.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
143402025-11-13T08:07:03.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143412025-11-13T08:07:03.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143422025-11-13T08:07:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143432025-11-13T08:07:03.487ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
143442025-11-13T08:07:03.487ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
143452025-11-13T08:07:03.488ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
143462025-11-13T08:07:03.489ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143472025-11-13T08:07:03.495ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
143482025-11-13T08:07:03.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143492025-11-13T08:07:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143502025-11-13T08:07:03.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143512025-11-13T08:07:03.499ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143522025-11-13T08:07:03.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143532025-11-13T08:07:03.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143542025-11-13T08:07:03.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143552025-11-13T08:07:03.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143562025-11-13T08:07:03.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143572025-11-13T08:07:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143582025-11-13T08:07:03.507ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143592025-11-13T08:07:03.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143602025-11-13T08:07:03.509ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143612025-11-13T08:07:03.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143622025-11-13T08:07:03.510ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143632025-11-13T08:07:03.511ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143642025-11-13T08:07:03.512ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143652025-11-13T08:07:03.517ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
143662025-11-13T08:07:03.517ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143672025-11-13T08:07:03.517ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143682025-11-13T08:07:03.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143692025-11-13T08:07:03.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143702025-11-13T08:07:03.521ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143712025-11-13T08:07:03.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143722025-11-13T08:07:03.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143732025-11-13T08:07:03.528ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
143742025-11-13T08:07:03.528ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
143752025-11-13T08:07:03.529ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b06c374-902b-4758-856f-1cc48a2fe44d", "content-length": "372", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
143762025-11-13T08:07:03.529ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
143772025-11-13T08:07:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
143782025-11-13T08:07:03.531ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
143792025-11-13T08:07:03.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143802025-11-13T08:07:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143812025-11-13T08:07:03.539ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
143822025-11-13T08:07:03.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143832025-11-13T08:07:03.540ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143842025-11-13T08:07:03.542ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143852025-11-13T08:07:03.544ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
143862025-11-13T08:07:03.544ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
143872025-11-13T08:07:03.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
143882025-11-13T08:07:03.546ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
143892025-11-13T08:07:03.547ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
143902025-11-13T08:07:03.547ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
143912025-11-13T08:07:03.575ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
143922025-11-13T08:07:03.576ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ab5dfc3-46c3-4781-9b69-9dc5cd7a159f", "content-length": "691", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
143932025-11-13T08:07:03.576ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
143942025-11-13T08:07:03.576ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
143952025-11-13T08:07:03.576ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
143962025-11-13T08:07:03.579ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 935.905469ms iteration = 17 status = {}
143972025-11-13T08:07:03.592ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
143982025-11-13T08:07:03.593ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "809ae04c-2cc1-4521-8fc0-42dee6917350", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
143992025-11-13T08:07:03.593ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
144002025-11-13T08:07:03.595ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "aa4bc168-a3f2-4b00-aa21-5d73bdbde4c0", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
144012025-11-13T08:07:03.595ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
144022025-11-13T08:07:03.597ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "ed20b363-5947-4841-be03-bb78459af506", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
144032025-11-13T08:07:03.597ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
144042025-11-13T08:07:03.598ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
144052025-11-13T08:07:03.598ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
144062025-11-13T08:07:03.598ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "c1ed72a0-e85a-49cf-ad09-589337029330", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
144072025-11-13T08:07:03.598ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
144082025-11-13T08:07:03.599ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
144092025-11-13T08:07:03.599ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
144102025-11-13T08:07:03.599ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 992.161305ms iteration = 11 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
144112025-11-13T08:07:03.601ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
144122025-11-13T08:07:03.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
144132025-11-13T08:07:03.604ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
144142025-11-13T08:07:03.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
144152025-11-13T08:07:03.608ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
144162025-11-13T08:07:03.608ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
144172025-11-13T08:07:03.612ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e2cdaf82-e10b-48f0-9b3a-276153d98314", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
144182025-11-13T08:07:03.613ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 215.932235ms iteration = 26 status = {}
144192025-11-13T08:07:03.700ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
144202025-11-13T08:07:03.700ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
144212025-11-13T08:07:03.734ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "38e01274-29ae-4c54-b182-eefa8da12627", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
144222025-11-13T08:07:03.734ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
144232025-11-13T08:07:03.734ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
144242025-11-13T08:07:03.734ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "81e0a91c-b2fe-478f-bdca-b8f63aa88c4e", "content-length": "682", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
144252025-11-13T08:07:03.735ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
144262025-11-13T08:07:03.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
144272025-11-13T08:07:03.735ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144282025-11-13T08:07:03.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144292025-11-13T08:07:03.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144302025-11-13T08:07:03.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144312025-11-13T08:07:03.739ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
144322025-11-13T08:07:03.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
144332025-11-13T08:07:03.741ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144342025-11-13T08:07:03.743ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
144352025-11-13T08:07:03.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
144362025-11-13T08:07:03.744ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
144372025-11-13T08:07:03.744ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
144382025-11-13T08:07:03.745ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "0cef7823-0c38-41c3-998b-7a1dbe52988d", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
144392025-11-13T08:07:03.745ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
144402025-11-13T08:07:03.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
144412025-11-13T08:07:03.746ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
144422025-11-13T08:07:03.746ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = Dns step id = Ensure
144432025-11-13T08:07:03.746ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
144442025-11-13T08:07:03.746ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144452025-11-13T08:07:03.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144462025-11-13T08:07:03.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144472025-11-13T08:07:03.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144482025-11-13T08:07:03.749ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144492025-11-13T08:07:03.752ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.109486722s iteration = 17 status = {}
144502025-11-13T08:07:03.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
144512025-11-13T08:07:03.754ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
144522025-11-13T08:07:03.754ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
144532025-11-13T08:07:03.754ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
144542025-11-13T08:07:03.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144552025-11-13T08:07:03.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144562025-11-13T08:07:03.755ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144572025-11-13T08:07:03.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
144582025-11-13T08:07:03.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144592025-11-13T08:07:03.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144602025-11-13T08:07:03.760ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
144612025-11-13T08:07:03.760ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
144622025-11-13T08:07:03.760ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144632025-11-13T08:07:03.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144642025-11-13T08:07:03.762ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144652025-11-13T08:07:03.766ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
144662025-11-13T08:07:03.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144672025-11-13T08:07:03.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144682025-11-13T08:07:03.768ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
144692025-11-13T08:07:03.768ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
144702025-11-13T08:07:03.768ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
144712025-11-13T08:07:03.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144722025-11-13T08:07:03.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144732025-11-13T08:07:03.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144742025-11-13T08:07:03.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144752025-11-13T08:07:03.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144762025-11-13T08:07:03.774ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
144772025-11-13T08:07:03.775ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
144782025-11-13T08:07:03.775ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
144792025-11-13T08:07:03.775ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
144802025-11-13T08:07:03.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144812025-11-13T08:07:03.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144822025-11-13T08:07:03.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144832025-11-13T08:07:03.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
144842025-11-13T08:07:03.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144852025-11-13T08:07:03.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144862025-11-13T08:07:03.820ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
144872025-11-13T08:07:03.821ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
144882025-11-13T08:07:03.821ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144892025-11-13T08:07:03.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144902025-11-13T08:07:03.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144912025-11-13T08:07:03.826ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
144922025-11-13T08:07:03.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144932025-11-13T08:07:03.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144942025-11-13T08:07:03.828ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
144952025-11-13T08:07:03.828ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
144962025-11-13T08:07:03.828ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
144972025-11-13T08:07:03.828ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
144982025-11-13T08:07:03.830ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144992025-11-13T08:07:03.832ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
145002025-11-13T08:07:03.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145012025-11-13T08:07:03.832ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145022025-11-13T08:07:03.834ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145032025-11-13T08:07:03.834ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145042025-11-13T08:07:03.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145052025-11-13T08:07:03.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145062025-11-13T08:07:03.836ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145072025-11-13T08:07:03.840ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
145082025-11-13T08:07:03.842ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
145092025-11-13T08:07:03.842ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
145102025-11-13T08:07:03.842ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = OmicronZones step id = Cleanup
145112025-11-13T08:07:03.842ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = OmicronZones step id = Cleanup
145122025-11-13T08:07:03.842ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = PhysicalDisks step id = Cleanup
145132025-11-13T08:07:03.842ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = Clickhouse step id = Ensure
145142025-11-13T08:07:03.842ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = Clickhouse step id = Ensure
145152025-11-13T08:07:03.873ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
145162025-11-13T08:07:03.877ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "de673c0e-9927-455d-a8cb-b892c0024136", "date": "Thu, 13 Nov 2025 08:07:03 GMT"} })
145172025-11-13T08:07:03.879ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = SupportBundles step id = Cleanup
145182025-11-13T08:07:03.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145192025-11-13T08:07:03.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145202025-11-13T08:07:03.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145212025-11-13T08:07:03.882ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
145222025-11-13T08:07:03.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145232025-11-13T08:07:03.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145242025-11-13T08:07:03.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145252025-11-13T08:07:03.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145262025-11-13T08:07:03.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145272025-11-13T08:07:03.888ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
145282025-11-13T08:07:03.892ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = OmicronZones step id = Cleanup
145292025-11-13T08:07:03.892ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
145302025-11-13T08:07:03.892ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
145312025-11-13T08:07:03.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145322025-11-13T08:07:03.892ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145332025-11-13T08:07:03.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145342025-11-13T08:07:03.895ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
145352025-11-13T08:07:03.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145362025-11-13T08:07:03.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145372025-11-13T08:07:03.932ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
145382025-11-13T08:07:03.933ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
145392025-11-13T08:07:03.933ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = Cockroach step id = Ensure
145402025-11-13T08:07:03.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
145412025-11-13T08:07:03.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
145422025-11-13T08:07:03.949ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
145432025-11-13T08:07:03.949ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = c32a326b-6928-4123-8867-b3f7307bff87 step component = MgsUpdates step id = Ensure
145442025-11-13T08:07:03.949ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Dependency
145452025-11-13T08:07:03.949ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145462025-11-13T08:07:03.949ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 26 reason = Dependency
145472025-11-13T08:07:03.949ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
145482025-11-13T08:07:03.950ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.341738137s iteration = 10 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"e6535089-af4d-411e-b59c-21c684672965","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3680,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":134232219,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":134232169,"secs":0}},"event_index":1,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":134256280,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":272793553,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":272793513,"secs":0}},"event_index":2,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":407074843,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":108093995,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":108093935,"secs":0}},"event_index":3,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":515191849,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":276132898,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":276132868,"secs":0}},"event_index":4,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":791346127,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":347284748,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":347284708,"secs":0}},"event_index":5,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":138642405,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":95288694,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":95288643,"secs":0}},"event_index":6,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":233943598,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":5330,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":5290,"secs":0}},"event_index":7,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":233964759,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2580,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2540,"secs":0}},"event_index":8,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":233970659,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3500,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3460,"secs":0}},"event_index":9,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":233976979,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2020,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1980,"secs":0}},"event_index":10,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":233984949,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36889886,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36889846,"secs":0}},"event_index":11,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":270876575,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":13251320,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":13251280,"secs":0}},"event_index":12,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":284137975,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":40510636,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":40510576,"secs":0}},"event_index":13,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":324656252,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":16735689,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":16735649,"secs":0}},"event_index":14,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":341401341,"secs":1}},{"data":{"attempt_elapsed":{"nanos":8031,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":7991,"secs":0}},"event_index":15,"execution_id":"e6535089-af4d-411e-b59c-21c684672965","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":341419602,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
145492025-11-13T08:07:03.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
145502025-11-13T08:07:03.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145512025-11-13T08:07:03.951ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145522025-11-13T08:07:03.952ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145532025-11-13T08:07:03.954ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
145542025-11-13T08:07:03.954ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
145552025-11-13T08:07:03.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
145562025-11-13T08:07:03.955ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
145572025-11-13T08:07:03.955ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145582025-11-13T08:07:03.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145592025-11-13T08:07:03.998ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
145602025-11-13T08:07:03.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
145612025-11-13T08:07:04.000ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145622025-11-13T08:07:04.002ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
145632025-11-13T08:07:04.002ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
145642025-11-13T08:07:04.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
145652025-11-13T08:07:04.004ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
145662025-11-13T08:07:04.004ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 54.502011ms iteration = 26 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
145672025-11-13T08:07:04.086ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(c3f9dc97-7cb9-4750-a624-009dc8a84400, 0), (e3c03c67-3c91-41f9-b321-adac53055e60, 0), (db0127fd-bd52-4625-be88-6185cc000069, 0), (4ba91f9a-a9b4-4c47-93bd-d51c608704f2, 0)] collection_id = c3f9dc97-7cb9-4750-a624-009dc8a84400
145682025-11-13T08:07:04.086ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145692025-11-13T08:07:04.086ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145702025-11-13T08:07:04.264ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = c3f9dc97-7cb9-4750-a624-009dc8a84400 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
145712025-11-13T08:07:04.264ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
145722025-11-13T08:07:04.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
145732025-11-13T08:07:04.268ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(e3c03c67-3c91-41f9-b321-adac53055e60, 0), (db0127fd-bd52-4625-be88-6185cc000069, 0), (4ba91f9a-a9b4-4c47-93bd-d51c608704f2, 0)]
145742025-11-13T08:07:04.269ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6432, tv_nsec: 621456183 } })
145752025-11-13T08:07:04.300ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6432, tv_nsec: 653153676 } })
145762025-11-13T08:07:04.301ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
145772025-11-13T08:07:04.301ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
145782025-11-13T08:07:04.301ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
145792025-11-13T08:07:04.302ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7af7141a-6beb-46b0-b34a-da8ab7572a1d", "content-length": "476", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145802025-11-13T08:07:04.302ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
145812025-11-13T08:07:04.302ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7855b753-3bad-4755-b002-b4320514cad8", "content-length": "734", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145822025-11-13T08:07:04.302ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
145832025-11-13T08:07:04.302ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d582fd3-b4e7-400c-8ee3-c2b7862a964b", "content-length": "111", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145842025-11-13T08:07:04.302ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
145852025-11-13T08:07:04.303ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5ab5fef-ba76-4c70-a940-47bdf8cc1ed2", "content-length": "111", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145862025-11-13T08:07:04.303ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
145872025-11-13T08:07:04.303ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38b035c1-1976-40d5-8cdb-3e5ee630aa07", "content-length": "163", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145882025-11-13T08:07:04.303ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
145892025-11-13T08:07:04.304ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b7af669-5c2a-4ad7-ade0-717cd4f94682", "content-length": "163", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145902025-11-13T08:07:04.304ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
145912025-11-13T08:07:04.305ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "08504890-f3ba-4e16-ae98-2768b0c4f1c9", "content-length": "166", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145922025-11-13T08:07:04.305ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
145932025-11-13T08:07:04.305ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5984603-55e5-47b6-8b5e-9c6af1771e5e", "content-length": "166", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145942025-11-13T08:07:04.305ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
145952025-11-13T08:07:04.305ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "38fac310-745e-474a-8257-fcff690bc38a", "content-length": "702", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145962025-11-13T08:07:04.306ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
145972025-11-13T08:07:04.306ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ae620e-8883-4cce-98a4-f34c23229102", "content-length": "727", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
145982025-11-13T08:07:04.306ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
145992025-11-13T08:07:04.306ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0826d382-7f75-4697-bd50-54d7499bcaf1", "content-length": "729", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146002025-11-13T08:07:04.306ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
146012025-11-13T08:07:04.307ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "900e9c05-9cdb-4353-82ba-bffc13c146ae", "content-length": "728", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146022025-11-13T08:07:04.307ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
146032025-11-13T08:07:04.307ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fab68fb8-59b4-4e0c-9618-a3f410dc1b04", "content-length": "722", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146042025-11-13T08:07:04.307ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
146052025-11-13T08:07:04.307ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "6df47f71-cd57-4346-95aa-5c11cd15d029", "content-length": "10", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146062025-11-13T08:07:04.307ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
146072025-11-13T08:07:04.308ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5763c35c-9934-4cb3-9583-ee8af3581e74", "content-length": "144", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146082025-11-13T08:07:04.308ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
146092025-11-13T08:07:04.308ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4784b877-446a-45f8-af56-aac7ea692333", "content-length": "147", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146102025-11-13T08:07:04.308ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
146112025-11-13T08:07:04.309ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4905ffbd-ef75-45c0-b7ba-c95713662b1c", "content-length": "109", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146122025-11-13T08:07:04.309ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
146132025-11-13T08:07:04.310ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dafbb95-5190-4f12-80c5-5709ea5ab5bf", "content-length": "109", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146142025-11-13T08:07:04.310ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
146152025-11-13T08:07:04.310ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cca9722-21f2-4b16-97fb-78241d062b90", "content-length": "163", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146162025-11-13T08:07:04.310ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
146172025-11-13T08:07:04.311ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3f62fda-177e-462f-b368-bd5d6d754d96", "content-length": "163", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146182025-11-13T08:07:04.311ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
146192025-11-13T08:07:04.311ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0442a1d9-ed90-4b45-a7e5-e210c58cdb78", "content-length": "166", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146202025-11-13T08:07:04.311ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
146212025-11-13T08:07:04.312ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd991744-95cd-463c-be35-0871f30c2b3d", "content-length": "166", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146222025-11-13T08:07:04.312ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
146232025-11-13T08:07:04.312ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e430c08-2834-4c66-bf39-f34fbc4d8982", "content-length": "702", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146242025-11-13T08:07:04.312ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
146252025-11-13T08:07:04.312ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0c526de-518f-44b5-9a17-8884e870cf02", "content-length": "727", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146262025-11-13T08:07:04.312ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
146272025-11-13T08:07:04.312ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae861217-f61f-427d-abd2-9a251cb6e601", "content-length": "729", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146282025-11-13T08:07:04.313ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
146292025-11-13T08:07:04.313ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "097987d2-7743-42b2-b16a-70c8f26d58a5", "content-length": "728", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146302025-11-13T08:07:04.313ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146312025-11-13T08:07:04.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
146322025-11-13T08:07:04.315ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146332025-11-13T08:07:04.317ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
146342025-11-13T08:07:04.317ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146352025-11-13T08:07:04.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
146362025-11-13T08:07:04.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146372025-11-13T08:07:04.321ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
146382025-11-13T08:07:04.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146392025-11-13T08:07:04.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
146402025-11-13T08:07:04.357ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
146412025-11-13T08:07:04.357ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
146422025-11-13T08:07:04.414ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "06b5c733-054c-4b11-af02-b1a354df5031", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146432025-11-13T08:07:04.416ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
146442025-11-13T08:07:04.416ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
146452025-11-13T08:07:04.416ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
146462025-11-13T08:07:04.416ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
146472025-11-13T08:07:04.416ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
146482025-11-13T08:07:04.416ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
146492025-11-13T08:07:04.416ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
146502025-11-13T08:07:04.416ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
146512025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a268d72c-161c-4b0e-a668-95d9886f682d", "content-length": "64", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146522025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "36892444-e2b9-4fb3-ab4e-9958e3cbe996", "content-length": "64", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146532025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
146542025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
146552025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "45390534-1180-445b-8a50-bd2c43cfd286", "content-length": "64", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146562025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "95f1cadb-fe76-49e6-a7b7-738c3f796440", "content-length": "64", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146572025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
146582025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "a62ab62b-723e-4cdb-86fc-184511528cb2", "content-length": "64", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146592025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
146602025-11-13T08:07:04.417ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
146612025-11-13T08:07:04.456ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecc1573d-2a2f-49fd-9c70-991e9c66aa64", "content-length": "399635", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146622025-11-13T08:07:04.457ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f7dd7c4-0400-44e2-a873-ce6da749e5ce", "content-length": "407164", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146632025-11-13T08:07:04.457ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9b92ba3-0580-4325-8805-0ebaac322222", "content-length": "401684", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146642025-11-13T08:07:04.461ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f355eb6d-66c2-4195-9db7-cd659da3df88", "content-length": "399427", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146652025-11-13T08:07:04.462ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6f753f5-acb0-4922-aec5-80f193e3c0c3", "content-length": "423996", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146662025-11-13T08:07:04.464ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
146672025-11-13T08:07:04.464ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
146682025-11-13T08:07:04.464ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
146692025-11-13T08:07:04.464ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
146702025-11-13T08:07:04.464ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
146712025-11-13T08:07:04.498ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
146722025-11-13T08:07:04.498ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
146732025-11-13T08:07:04.501ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "edd93306-97a9-41d4-bb71-a3415555909c", "content-length": "125", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146742025-11-13T08:07:04.503ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
146752025-11-13T08:07:04.604ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
146762025-11-13T08:07:04.604ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
146772025-11-13T08:07:04.605ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ef7187b-116a-4820-a721-dbfd18094459", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146782025-11-13T08:07:04.605ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
146792025-11-13T08:07:04.607ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
146802025-11-13T08:07:04.608ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
146812025-11-13T08:07:04.608ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cd5fa8f-e284-47e8-b7bd-7a9d791e7185", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146822025-11-13T08:07:04.608ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
146832025-11-13T08:07:04.610ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
146842025-11-13T08:07:04.610ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
146852025-11-13T08:07:04.611ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "42f9a095-dc85-49bf-bb69-2937b238caec", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:07:04 GMT"} })
146862025-11-13T08:07:04.611ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
146872025-11-13T08:07:04.614ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
146882025-11-13T08:07:04.614ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
146892025-11-13T08:07:04.616ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146902025-11-13T08:07:04.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
146912025-11-13T08:07:04.619ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146922025-11-13T08:07:04.622ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
146932025-11-13T08:07:04.622ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146942025-11-13T08:07:04.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
146952025-11-13T08:07:04.857ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 5588c052-6347-48b0-b6e1-745a625f895b file = nexus/db-queries/src/db/datastore/inventory.rs:1613
146962025-11-13T08:07:04.857ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 5588c052-6347-48b0-b6e1-745a625f895b time_started = 2025-11-13 08:07:04.301546 UTC
146972025-11-13T08:07:04.857ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 43 reason = Dependency
146982025-11-13T08:07:04.857ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
146992025-11-13T08:07:04.857ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 907.874878ms iteration = 10 status = {"collection_id":"5588c052-6347-48b0-b6e1-745a625f895b","time_done":"2025-11-13 08:07:04.614081 UTC","time_started":"2025-11-13 08:07:04.301546 UTC"}
147002025-11-13T08:07:04.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
147012025-11-13T08:07:04.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147022025-11-13T08:07:04.862ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
147032025-11-13T08:07:04.862ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
147042025-11-13T08:07:04.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
147052025-11-13T08:07:04.903ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
147062025-11-13T08:07:04.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
147072025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 22 reason = Dependency
147082025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 96.715952ms iteration = 43 status = {"Loaded":{"collection_id":"484d8991-1076-40d9-9a86-ae1c48054b42","time_loaded":"2025-11-13T08:07:04.857665256Z","time_started":"2025-11-13T08:07:04.309354Z"}}
147092025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 13 reason = Dependency
147102025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 28 reason = Dependency
147112025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147122025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
147132025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147142025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147152025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147162025-11-13T08:07:04.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147172025-11-13T08:07:04.956ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147182025-11-13T08:07:04.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147192025-11-13T08:07:04.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147202025-11-13T08:07:04.959ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
147212025-11-13T08:07:04.959ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
147222025-11-13T08:07:04.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147232025-11-13T08:07:04.959ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
147242025-11-13T08:07:04.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147252025-11-13T08:07:04.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147262025-11-13T08:07:04.960ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
147272025-11-13T08:07:04.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147282025-11-13T08:07:04.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147292025-11-13T08:07:04.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147302025-11-13T08:07:04.961ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
147312025-11-13T08:07:04.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147322025-11-13T08:07:04.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147332025-11-13T08:07:04.962ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147342025-11-13T08:07:04.963ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
147352025-11-13T08:07:04.963ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147362025-11-13T08:07:04.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147372025-11-13T08:07:04.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147382025-11-13T08:07:04.965ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
147392025-11-13T08:07:04.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147402025-11-13T08:07:04.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147412025-11-13T08:07:04.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147422025-11-13T08:07:04.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
147432025-11-13T08:07:04.966ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
147442025-11-13T08:07:04.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
147452025-11-13T08:07:04.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147462025-11-13T08:07:04.968ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147472025-11-13T08:07:04.968ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147482025-11-13T08:07:04.969ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
147492025-11-13T08:07:04.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147502025-11-13T08:07:04.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147512025-11-13T08:07:04.970ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147522025-11-13T08:07:04.971ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147532025-11-13T08:07:04.972ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
147542025-11-13T08:07:04.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147552025-11-13T08:07:04.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147562025-11-13T08:07:04.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147572025-11-13T08:07:04.975ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
147582025-11-13T08:07:04.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147592025-11-13T08:07:04.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147602025-11-13T08:07:04.976ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
147612025-11-13T08:07:04.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147622025-11-13T08:07:04.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147632025-11-13T08:07:04.978ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
147642025-11-13T08:07:04.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147652025-11-13T08:07:04.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147662025-11-13T08:07:04.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147672025-11-13T08:07:04.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147682025-11-13T08:07:04.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147692025-11-13T08:07:04.981ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
147702025-11-13T08:07:04.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147712025-11-13T08:07:04.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147722025-11-13T08:07:04.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147732025-11-13T08:07:04.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147742025-11-13T08:07:04.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147752025-11-13T08:07:04.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147762025-11-13T08:07:04.986ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
147772025-11-13T08:07:04.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
147782025-11-13T08:07:04.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
147792025-11-13T08:07:04.987ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147802025-11-13T08:07:04.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147812025-11-13T08:07:04.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147822025-11-13T08:07:04.988ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
147832025-11-13T08:07:04.988ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 34.172082ms iteration = 13 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"484d8991-1076-40d9-9a86-ae1c48054b42","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
147842025-11-13T08:07:04.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147852025-11-13T08:07:04.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147862025-11-13T08:07:04.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147872025-11-13T08:07:04.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
147882025-11-13T08:07:04.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147892025-11-13T08:07:04.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147902025-11-13T08:07:04.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147912025-11-13T08:07:04.999ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
147922025-11-13T08:07:04.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147932025-11-13T08:07:04.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147942025-11-13T08:07:05.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147952025-11-13T08:07:05.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
147962025-11-13T08:07:05.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147972025-11-13T08:07:05.005ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
147982025-11-13T08:07:05.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
147992025-11-13T08:07:05.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148002025-11-13T08:07:05.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148012025-11-13T08:07:05.010ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148022025-11-13T08:07:05.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148032025-11-13T08:07:05.012ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 58.090472ms iteration = 28 status = {"physical_disks_added":0}
148042025-11-13T08:07:05.015ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
148052025-11-13T08:07:05.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148062025-11-13T08:07:05.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148072025-11-13T08:07:05.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148082025-11-13T08:07:05.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148092025-11-13T08:07:05.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148102025-11-13T08:07:05.022ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
148112025-11-13T08:07:05.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148122025-11-13T08:07:05.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148132025-11-13T08:07:05.040ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148142025-11-13T08:07:05.040ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148152025-11-13T08:07:05.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148162025-11-13T08:07:05.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148172025-11-13T08:07:05.043ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148182025-11-13T08:07:05.046ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
148192025-11-13T08:07:05.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148202025-11-13T08:07:05.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148212025-11-13T08:07:05.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148222025-11-13T08:07:05.051ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
148232025-11-13T08:07:05.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148242025-11-13T08:07:05.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148252025-11-13T08:07:05.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148262025-11-13T08:07:05.054ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148272025-11-13T08:07:05.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148282025-11-13T08:07:05.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148292025-11-13T08:07:05.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148302025-11-13T08:07:05.061ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
148312025-11-13T08:07:05.061ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148322025-11-13T08:07:05.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148332025-11-13T08:07:05.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148342025-11-13T08:07:05.069ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
148352025-11-13T08:07:05.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148362025-11-13T08:07:05.069ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148372025-11-13T08:07:05.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148382025-11-13T08:07:05.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148392025-11-13T08:07:05.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148402025-11-13T08:07:05.074ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148412025-11-13T08:07:05.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148422025-11-13T08:07:05.079ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
148432025-11-13T08:07:05.112ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148442025-11-13T08:07:05.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148452025-11-13T08:07:05.113ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148462025-11-13T08:07:05.114ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148472025-11-13T08:07:05.115ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148482025-11-13T08:07:05.119ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
148492025-11-13T08:07:05.120ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148502025-11-13T08:07:05.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148512025-11-13T08:07:05.123ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148522025-11-13T08:07:05.123ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148532025-11-13T08:07:05.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148542025-11-13T08:07:05.128ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
148552025-11-13T08:07:05.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148562025-11-13T08:07:05.128ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148572025-11-13T08:07:05.130ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148582025-11-13T08:07:05.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148592025-11-13T08:07:05.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148602025-11-13T08:07:05.134ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
148612025-11-13T08:07:05.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148622025-11-13T08:07:05.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148632025-11-13T08:07:05.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148642025-11-13T08:07:05.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148652025-11-13T08:07:05.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148662025-11-13T08:07:05.141ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
148672025-11-13T08:07:05.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148682025-11-13T08:07:05.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148692025-11-13T08:07:05.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148702025-11-13T08:07:05.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148712025-11-13T08:07:05.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148722025-11-13T08:07:05.146ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
148732025-11-13T08:07:05.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148742025-11-13T08:07:05.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148752025-11-13T08:07:05.148ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
148762025-11-13T08:07:05.148ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
148772025-11-13T08:07:05.148ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 94dc05f1-3f5b-44f2-a14d-9fa46df74af3 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
148782025-11-13T08:07:05.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148792025-11-13T08:07:05.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148802025-11-13T08:07:05.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148812025-11-13T08:07:05.152ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
148822025-11-13T08:07:05.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
148832025-11-13T08:07:05.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
148842025-11-13T08:07:05.155ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
148852025-11-13T08:07:05.155ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
148862025-11-13T08:07:05.155ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 201.396879ms iteration = 22 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
148872025-11-13T08:07:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 276 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 00ccc8bb-6869-4dee-a3f5-3570a2f595bf response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
148882025-11-13T08:07:07.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 98 reason = Timeout
148892025-11-13T08:07:07.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148902025-11-13T08:07:07.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148912025-11-13T08:07:07.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148922025-11-13T08:07:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
148932025-11-13T08:07:07.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
148942025-11-13T08:07:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
148952025-11-13T08:07:07.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
148962025-11-13T08:07:07.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.031388ms iteration = 98 status = {"config_updated":false}
148972025-11-13T08:07:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 50 reason = Timeout
148982025-11-13T08:07:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 99 reason = Timeout
148992025-11-13T08:07:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149002025-11-13T08:07:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149012025-11-13T08:07:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149022025-11-13T08:07:12.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149032025-11-13T08:07:12.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149042025-11-13T08:07:12.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149052025-11-13T08:07:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
149062025-11-13T08:07:12.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149072025-11-13T08:07:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149082025-11-13T08:07:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
149092025-11-13T08:07:12.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149102025-11-13T08:07:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149112025-11-13T08:07:12.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
149122025-11-13T08:07:12.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.401565ms iteration = 99 status = {"config_updated":false}
149132025-11-13T08:07:12.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149142025-11-13T08:07:12.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149152025-11-13T08:07:12.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149162025-11-13T08:07:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
149172025-11-13T08:07:12.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149182025-11-13T08:07:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149192025-11-13T08:07:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 50 reason = Timeout
149202025-11-13T08:07:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
149212025-11-13T08:07:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
149222025-11-13T08:07:12.655ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
149232025-11-13T08:07:12.655ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.291936ms iteration = 50 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
149242025-11-13T08:07:12.679ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
149252025-11-13T08:07:12.679ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
149262025-11-13T08:07:12.720ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 78.371841ms iteration = 50 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
149272025-11-13T08:07:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 273 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 99aef6a7-f44c-49a6-890c-4cfccbedcc3b response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
149282025-11-13T08:07:17.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 44 reason = Timeout
149292025-11-13T08:07:17.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 100 reason = Timeout
149302025-11-13T08:07:17.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
149312025-11-13T08:07:17.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149322025-11-13T08:07:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
149332025-11-13T08:07:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149342025-11-13T08:07:17.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149352025-11-13T08:07:17.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149362025-11-13T08:07:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
149372025-11-13T08:07:17.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
149382025-11-13T08:07:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
149392025-11-13T08:07:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
149402025-11-13T08:07:17.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149412025-11-13T08:07:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149422025-11-13T08:07:17.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 484d8991-1076-40d9-9a86-ae1c48054b42 original_time_started = 2025-11-13 08:07:04.309354 UTC
149432025-11-13T08:07:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.278602ms iteration = 44 status = {"Loaded":{"collection_id":"484d8991-1076-40d9-9a86-ae1c48054b42","time_loaded":"2025-11-13T08:07:17.626445482Z","time_started":"2025-11-13T08:07:04.309354Z"}}
149442025-11-13T08:07:17.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
149452025-11-13T08:07:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.401556ms iteration = 100 status = {"config_updated":false}
149462025-11-13T08:07:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 34 reason = Timeout
149472025-11-13T08:07:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 34 reason = Timeout
149482025-11-13T08:07:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
149492025-11-13T08:07:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
149502025-11-13T08:07:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
149512025-11-13T08:07:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
149522025-11-13T08:07:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
149532025-11-13T08:07:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
149542025-11-13T08:07:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149552025-11-13T08:07:17.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149562025-11-13T08:07:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
149572025-11-13T08:07:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
149582025-11-13T08:07:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
149592025-11-13T08:07:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
149602025-11-13T08:07:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.61745ms iteration = 34 status = "NoSitrep"
149612025-11-13T08:07:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
149622025-11-13T08:07:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
149632025-11-13T08:07:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
149642025-11-13T08:07:17.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
149652025-11-13T08:07:17.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149662025-11-13T08:07:17.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
149672025-11-13T08:07:17.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
149682025-11-13T08:07:17.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.301452ms iteration = 34 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
149692025-11-13T08:07:22.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 51 reason = Timeout
149702025-11-13T08:07:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 101 reason = Timeout
149712025-11-13T08:07:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149722025-11-13T08:07:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149732025-11-13T08:07:22.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149742025-11-13T08:07:22.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149752025-11-13T08:07:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149762025-11-13T08:07:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149772025-11-13T08:07:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
149782025-11-13T08:07:22.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
149792025-11-13T08:07:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
149802025-11-13T08:07:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
149812025-11-13T08:07:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149822025-11-13T08:07:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149832025-11-13T08:07:22.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
149842025-11-13T08:07:22.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.508378ms iteration = 101 status = {"config_updated":false}
149852025-11-13T08:07:22.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149862025-11-13T08:07:22.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149872025-11-13T08:07:22.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149882025-11-13T08:07:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
149892025-11-13T08:07:22.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
149902025-11-13T08:07:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
149912025-11-13T08:07:22.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 51 reason = Timeout
149922025-11-13T08:07:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
149932025-11-13T08:07:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
149942025-11-13T08:07:22.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
149952025-11-13T08:07:22.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
149962025-11-13T08:07:22.655ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
149972025-11-13T08:07:22.655ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 29.271325ms iteration = 51 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
149982025-11-13T08:07:22.686ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 44.375592ms iteration = 51 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
149992025-11-13T08:07:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 462 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 5a7ca9e4-ce05-4909-a06d-89a0d62d7f28 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
150002025-11-13T08:07:27.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 102 reason = Timeout
150012025-11-13T08:07:27.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
150022025-11-13T08:07:27.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
150032025-11-13T08:07:27.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150042025-11-13T08:07:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
150052025-11-13T08:07:27.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
150062025-11-13T08:07:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
150072025-11-13T08:07:27.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
150082025-11-13T08:07:27.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.327535ms iteration = 102 status = {"config_updated":false}
150092025-11-13T08:07:32.606ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 18 reason = Timeout
150102025-11-13T08:07:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 18 reason = Timeout
150112025-11-13T08:07:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 18 reason = Timeout
150122025-11-13T08:07:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
150132025-11-13T08:07:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
150142025-11-13T08:07:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
150152025-11-13T08:07:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
150162025-11-13T08:07:32.610ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6460, tv_nsec: 962459674 } })
150172025-11-13T08:07:32.610ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
150182025-11-13T08:07:32.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6460, tv_nsec: 963036526 } })
150192025-11-13T08:07:32.618ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.986275ms iteration = 18 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
150202025-11-13T08:07:32.625ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 27 reason = Timeout
150212025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 45 reason = Timeout
150222025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
150232025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 29 reason = Timeout
150242025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
150252025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
150262025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 52 reason = Timeout
150272025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 103 reason = Timeout
150282025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
150292025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
150302025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
150312025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
150322025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
150332025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
150342025-11-13T08:07:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
150352025-11-13T08:07:32.630ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150362025-11-13T08:07:32.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150372025-11-13T08:07:32.630ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150382025-11-13T08:07:32.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150392025-11-13T08:07:32.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150402025-11-13T08:07:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
150412025-11-13T08:07:32.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
150422025-11-13T08:07:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
150432025-11-13T08:07:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
150442025-11-13T08:07:32.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
150452025-11-13T08:07:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
150462025-11-13T08:07:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
150472025-11-13T08:07:32.636ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
150482025-11-13T08:07:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
150492025-11-13T08:07:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
150502025-11-13T08:07:32.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
150512025-11-13T08:07:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
150522025-11-13T08:07:32.639ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
150532025-11-13T08:07:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
150542025-11-13T08:07:32.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
150552025-11-13T08:07:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
150562025-11-13T08:07:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
150572025-11-13T08:07:32.640ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
150582025-11-13T08:07:32.640ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.147628ms iteration = 103 status = {"config_updated":false}
150592025-11-13T08:07:32.641ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150602025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
150612025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 484d8991-1076-40d9-9a86-ae1c48054b42 original_time_started = 2025-11-13 08:07:04.309354 UTC
150622025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 16.408292ms iteration = 45 status = {"Loaded":{"collection_id":"484d8991-1076-40d9-9a86-ae1c48054b42","time_loaded":"2025-11-13T08:07:32.625984179Z","time_started":"2025-11-13T08:07:04.309354Z"}}
150632025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
150642025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 18 reason = Timeout
150652025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 27 reason = Timeout
150662025-11-13T08:07:32.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
150672025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
150682025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
150692025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 18 reason = Timeout
150702025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 18 reason = Timeout
150712025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
150722025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
150732025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 18 reason = Timeout
150742025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
150752025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
150762025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 18 reason = Timeout
150772025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
150782025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 52 reason = Timeout
150792025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
150802025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 18 reason = Timeout
150812025-11-13T08:07:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
150822025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
150832025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
150842025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
150852025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
150862025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
150872025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
150882025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
150892025-11-13T08:07:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
150902025-11-13T08:07:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
150912025-11-13T08:07:32.645ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
150922025-11-13T08:07:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
150932025-11-13T08:07:32.645ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150942025-11-13T08:07:32.646ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150952025-11-13T08:07:32.647ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150962025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 18 reason = Timeout
150972025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
150982025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 18 reason = Timeout
150992025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
151002025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 18 reason = Timeout
151012025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
151022025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
151032025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 18 reason = Timeout
151042025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
151052025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 35 reason = Timeout
151062025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
151072025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 18 reason = Timeout
151082025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
151092025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
151102025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 18 reason = Timeout
151112025-11-13T08:07:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
151122025-11-13T08:07:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
151132025-11-13T08:07:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
151142025-11-13T08:07:32.649ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 35 reason = Timeout
151152025-11-13T08:07:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
151162025-11-13T08:07:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
151172025-11-13T08:07:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
151182025-11-13T08:07:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6461, tv_nsec: 2054893 } })
151192025-11-13T08:07:32.650ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
151202025-11-13T08:07:32.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
151212025-11-13T08:07:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
151222025-11-13T08:07:32.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
151232025-11-13T08:07:32.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
151242025-11-13T08:07:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
151252025-11-13T08:07:32.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151262025-11-13T08:07:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
151272025-11-13T08:07:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
151282025-11-13T08:07:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151292025-11-13T08:07:32.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151302025-11-13T08:07:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
151312025-11-13T08:07:32.652ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
151322025-11-13T08:07:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
151332025-11-13T08:07:32.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151342025-11-13T08:07:32.653ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151352025-11-13T08:07:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
151362025-11-13T08:07:32.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
151372025-11-13T08:07:32.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
151382025-11-13T08:07:32.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
151392025-11-13T08:07:32.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
151402025-11-13T08:07:32.659ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
151412025-11-13T08:07:32.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
151422025-11-13T08:07:32.668ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6461, tv_nsec: 20342632 } })
151432025-11-13T08:07:32.669ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
151442025-11-13T08:07:32.671ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 28.675684ms iteration = 18 status = 0
151452025-11-13T08:07:32.673ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
151462025-11-13T08:07:32.673ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
151472025-11-13T08:07:32.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
151482025-11-13T08:07:32.674ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
151492025-11-13T08:07:32.674ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
151502025-11-13T08:07:32.674ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 31.851886ms iteration = 18 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
151512025-11-13T08:07:32.676ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
151522025-11-13T08:07:32.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
151532025-11-13T08:07:32.679ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151542025-11-13T08:07:32.682ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
151552025-11-13T08:07:32.683ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
151562025-11-13T08:07:32.683ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 34.460478ms iteration = 35 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
151572025-11-13T08:07:32.707ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
151582025-11-13T08:07:32.707ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 81.560024ms iteration = 52 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
151592025-11-13T08:07:32.714ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
151602025-11-13T08:07:32.714ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
151612025-11-13T08:07:32.715ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "42dac402-4df5-49a0-89c4-904fb1d9388c", "content-length": "26", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
151622025-11-13T08:07:32.715ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
151632025-11-13T08:07:32.718ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
151642025-11-13T08:07:32.719ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151652025-11-13T08:07:32.721ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
151662025-11-13T08:07:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
151672025-11-13T08:07:32.722ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
151682025-11-13T08:07:32.722ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
151692025-11-13T08:07:32.722ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
151702025-11-13T08:07:32.723ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
151712025-11-13T08:07:32.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
151722025-11-13T08:07:32.724ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.500122ms iteration = 52 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
151732025-11-13T08:07:32.725ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151742025-11-13T08:07:32.729ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
151752025-11-13T08:07:32.729ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
151762025-11-13T08:07:32.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
151772025-11-13T08:07:32.730ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151782025-11-13T08:07:32.731ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
151792025-11-13T08:07:32.731ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 105.642896ms iteration = 27 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
151802025-11-13T08:07:32.731ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 83.838988ms iteration = 18 status = {"errors":[],"finish_invoked_ok":[]}
151812025-11-13T08:07:32.735ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
151822025-11-13T08:07:32.735ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
151832025-11-13T08:07:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
151842025-11-13T08:07:32.737ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
151852025-11-13T08:07:32.737ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
151862025-11-13T08:07:32.737ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
151872025-11-13T08:07:32.737ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 89.305426ms iteration = 35 status = "NoSitrep"
151882025-11-13T08:07:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
151892025-11-13T08:07:32.740ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
151902025-11-13T08:07:32.740ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
151912025-11-13T08:07:32.740ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 91.757834ms iteration = 18 status = {"errors":[],"requests_created_ok":[]}
151922025-11-13T08:07:32.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
151932025-11-13T08:07:32.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
151942025-11-13T08:07:32.741ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
151952025-11-13T08:07:32.741ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
151962025-11-13T08:07:32.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
151972025-11-13T08:07:32.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
151982025-11-13T08:07:32.742ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 100.059217ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
151992025-11-13T08:07:32.743ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
152002025-11-13T08:07:32.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
152012025-11-13T08:07:32.744ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 96.1439ms iteration = 18 status = {"errors":[],"garbage_collect_requested":[]}
152022025-11-13T08:07:32.746ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 98.085859ms iteration = 18 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
152032025-11-13T08:07:32.749ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
152042025-11-13T08:07:32.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
152052025-11-13T08:07:32.761ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
152062025-11-13T08:07:32.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
152072025-11-13T08:07:32.763ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152082025-11-13T08:07:32.765ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
152092025-11-13T08:07:32.765ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
152102025-11-13T08:07:32.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
152112025-11-13T08:07:32.767ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152122025-11-13T08:07:32.769ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
152132025-11-13T08:07:32.769ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d28b76a-5c58-4fe3-85fd-a5597426f978", "content-length": "78", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152142025-11-13T08:07:32.769ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
152152025-11-13T08:07:32.769ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
152162025-11-13T08:07:32.769ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
152172025-11-13T08:07:32.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
152182025-11-13T08:07:32.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
152192025-11-13T08:07:32.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
152202025-11-13T08:07:32.771ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
152212025-11-13T08:07:32.771ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
152222025-11-13T08:07:32.772ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
152232025-11-13T08:07:32.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
152242025-11-13T08:07:32.772ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
152252025-11-13T08:07:32.773ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
152262025-11-13T08:07:32.776ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
152272025-11-13T08:07:32.776ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
152282025-11-13T08:07:32.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
152292025-11-13T08:07:32.776ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
152302025-11-13T08:07:32.776ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
152312025-11-13T08:07:32.777ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
152322025-11-13T08:07:32.777ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
152332025-11-13T08:07:32.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
152342025-11-13T08:07:32.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
152352025-11-13T08:07:32.777ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
152362025-11-13T08:07:32.779ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
152372025-11-13T08:07:32.779ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
152382025-11-13T08:07:32.780ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebed14af-4e3e-4aa9-8e46-6838858366c1", "content-length": "81", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152392025-11-13T08:07:32.780ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 173.647514ms iteration = 18 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
152402025-11-13T08:07:32.789ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 141.752117ms iteration = 18 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
152412025-11-13T08:07:32.806ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
152422025-11-13T08:07:32.806ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
152432025-11-13T08:07:32.806ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b52194de-bd5f-43ec-99fb-f0703290c21d", "content-length": "26", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152442025-11-13T08:07:32.806ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
152452025-11-13T08:07:32.808ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
152462025-11-13T08:07:32.816ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
152472025-11-13T08:07:32.816ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
152482025-11-13T08:07:32.816ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 173.995371ms iteration = 18 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
152492025-11-13T08:07:32.834ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
152502025-11-13T08:07:32.834ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
152512025-11-13T08:07:32.834ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "d568bde9-e377-4cf5-b492-8fd7eb77bcda", "content-length": "81", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152522025-11-13T08:07:32.837ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
152532025-11-13T08:07:32.837ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
152542025-11-13T08:07:32.837ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "ead8b466-0efa-4a5e-a455-4298e77fb2d1", "content-length": "81", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152552025-11-13T08:07:32.840ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 191.730089ms iteration = 18 status = {"disabled":false,"errors":[],"sps":[]}
152562025-11-13T08:07:32.853ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 227.237428ms iteration = 29 status = {"physical_disks_added":0}
152572025-11-13T08:07:32.878ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
152582025-11-13T08:07:32.878ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
152592025-11-13T08:07:32.879ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "35975b67-db23-4072-8d41-41b3b3855c4e", "content-length": "372", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152602025-11-13T08:07:32.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
152612025-11-13T08:07:32.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
152622025-11-13T08:07:32.893ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 286.095894ms iteration = 18 status = {}
152632025-11-13T08:07:32.900ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
152642025-11-13T08:07:32.900ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
152652025-11-13T08:07:32.900ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "103fa11c-efca-41e1-bc65-795d53dc1419", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152662025-11-13T08:07:32.900ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
152672025-11-13T08:07:32.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
152682025-11-13T08:07:32.906ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
152692025-11-13T08:07:32.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
152702025-11-13T08:07:32.909ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6461, tv_nsec: 261695057 } })
152712025-11-13T08:07:32.909ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
152722025-11-13T08:07:32.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
152732025-11-13T08:07:32.913ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
152742025-11-13T08:07:32.913ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
152752025-11-13T08:07:32.915ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "be241940-0d49-4343-bd49-c311e2720ce3", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152762025-11-13T08:07:32.916ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 274.084829ms iteration = 27 status = {}
152772025-11-13T08:07:32.927ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
152782025-11-13T08:07:32.932ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "092a59d1-7537-4edd-ac66-7d4a9e56893a", "content-length": "691", "date": "Thu, 13 Nov 2025 08:07:32 GMT"} })
152792025-11-13T08:07:32.932ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
152802025-11-13T08:07:32.932ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
152812025-11-13T08:07:32.932ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
152822025-11-13T08:07:32.934ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 292.087113ms iteration = 18 status = {}
152832025-11-13T08:07:32.979ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
152842025-11-13T08:07:32.979ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
152852025-11-13T08:07:32.981ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
152862025-11-13T08:07:32.981ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
152872025-11-13T08:07:32.982ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
152882025-11-13T08:07:32.994ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
152892025-11-13T08:07:32.994ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
152902025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
152912025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
152922025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152932025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152942025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152952025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
152962025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
152972025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
152982025-11-13T08:07:32.995ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
152992025-11-13T08:07:32.995ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153002025-11-13T08:07:32.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153012025-11-13T08:07:32.998ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
153022025-11-13T08:07:32.998ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
153032025-11-13T08:07:33.229ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "876ecc2c-611f-4d88-b7f6-8a64041672d1", "date": "Thu, 13 Nov 2025 08:07:33 GMT"} })
153042025-11-13T08:07:33.229ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
153052025-11-13T08:07:33.229ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
153062025-11-13T08:07:33.230ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e82473cb-846c-4671-bbe9-ba50d1d57dd7", "content-length": "682", "date": "Thu, 13 Nov 2025 08:07:33 GMT"} })
153072025-11-13T08:07:33.230ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153082025-11-13T08:07:33.230ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153092025-11-13T08:07:33.234ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153102025-11-13T08:07:33.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153112025-11-13T08:07:33.237ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153122025-11-13T08:07:33.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153132025-11-13T08:07:33.239ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
153142025-11-13T08:07:33.240ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
153152025-11-13T08:07:33.247ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 604.998624ms iteration = 18 status = {}
153162025-11-13T08:07:33.591ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 3
153172025-11-13T08:07:33.591ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
153182025-11-13T08:07:33.622ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
153192025-11-13T08:07:33.623ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:40620
153202025-11-13T08:07:33.623ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:40620 req_id = ba3509b5-44f2-4650-baf6-c6dd252b7d55 uri = /metrics/producers version = 1.0.0
153212025-11-13T08:07:33.623ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:40620 req_id = ba3509b5-44f2-4650-baf6-c6dd252b7d55 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
153222025-11-13T08:07:33.623ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:40620 req_id = ba3509b5-44f2-4650-baf6-c6dd252b7d55 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
153232025-11-13T08:07:33.662ZINFOnexus: assigned collector to new producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d file = nexus/src/app/oximeter.rs:76 producer_id = bcb58129-79cc-406c-aa16-455a5cd9f031
153242025-11-13T08:07:33.662ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 39089 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:40620 req_id = ba3509b5-44f2-4650-baf6-c6dd252b7d55 response_code = 201 uri = /metrics/producers
153252025-11-13T08:07:33.662ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "ba3509b5-44f2-4650-baf6-c6dd252b7d55", "content-length": "41", "date": "Thu, 13 Nov 2025 08:07:33 GMT"} })
153262025-11-13T08:07:33.663ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
153272025-11-13T08:07:35.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 448 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 9e09e148-a46e-4373-8a0c-82b057341d43 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
153282025-11-13T08:07:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 104 reason = Timeout
153292025-11-13T08:07:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153302025-11-13T08:07:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153312025-11-13T08:07:37.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153322025-11-13T08:07:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
153332025-11-13T08:07:37.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153342025-11-13T08:07:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153352025-11-13T08:07:37.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
153362025-11-13T08:07:37.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.14778ms iteration = 104 status = {"config_updated":false}
153372025-11-13T08:07:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 53 reason = Timeout
153382025-11-13T08:07:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 105 reason = Timeout
153392025-11-13T08:07:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153402025-11-13T08:07:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153412025-11-13T08:07:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
153422025-11-13T08:07:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153432025-11-13T08:07:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153442025-11-13T08:07:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153452025-11-13T08:07:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
153462025-11-13T08:07:42.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153472025-11-13T08:07:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
153482025-11-13T08:07:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
153492025-11-13T08:07:42.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153502025-11-13T08:07:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153512025-11-13T08:07:42.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153522025-11-13T08:07:42.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
153532025-11-13T08:07:42.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
153542025-11-13T08:07:42.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.544278ms iteration = 105 status = {"config_updated":false}
153552025-11-13T08:07:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153562025-11-13T08:07:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
153572025-11-13T08:07:42.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
153582025-11-13T08:07:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
153592025-11-13T08:07:42.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 53 reason = Timeout
153602025-11-13T08:07:42.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
153612025-11-13T08:07:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
153622025-11-13T08:07:42.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
153632025-11-13T08:07:42.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
153642025-11-13T08:07:42.651ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
153652025-11-13T08:07:42.651ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 25.4313ms iteration = 53 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
153662025-11-13T08:07:42.682ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 41.018767ms iteration = 53 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
153672025-11-13T08:07:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 301 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 59da3e6e-5cfc-4a36-bf11-d3c954587b56 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
153682025-11-13T08:07:47.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 46 reason = Timeout
153692025-11-13T08:07:47.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 106 reason = Timeout
153702025-11-13T08:07:47.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
153712025-11-13T08:07:47.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153722025-11-13T08:07:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
153732025-11-13T08:07:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153742025-11-13T08:07:47.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153752025-11-13T08:07:47.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153762025-11-13T08:07:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
153772025-11-13T08:07:47.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
153782025-11-13T08:07:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
153792025-11-13T08:07:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
153802025-11-13T08:07:47.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
153812025-11-13T08:07:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
153822025-11-13T08:07:47.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
153832025-11-13T08:07:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.735672ms iteration = 106 status = {"config_updated":false}
153842025-11-13T08:07:47.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 484d8991-1076-40d9-9a86-ae1c48054b42 original_time_started = 2025-11-13 08:07:04.309354 UTC
153852025-11-13T08:07:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.196941ms iteration = 46 status = {"Loaded":{"collection_id":"484d8991-1076-40d9-9a86-ae1c48054b42","time_loaded":"2025-11-13T08:07:47.626313187Z","time_started":"2025-11-13T08:07:04.309354Z"}}
153862025-11-13T08:07:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 36 reason = Timeout
153872025-11-13T08:07:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 36 reason = Timeout
153882025-11-13T08:07:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
153892025-11-13T08:07:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
153902025-11-13T08:07:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
153912025-11-13T08:07:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
153922025-11-13T08:07:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
153932025-11-13T08:07:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
153942025-11-13T08:07:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153952025-11-13T08:07:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153962025-11-13T08:07:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
153972025-11-13T08:07:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
153982025-11-13T08:07:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
153992025-11-13T08:07:47.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
154002025-11-13T08:07:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.976118ms iteration = 36 status = "NoSitrep"
154012025-11-13T08:07:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
154022025-11-13T08:07:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
154032025-11-13T08:07:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
154042025-11-13T08:07:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
154052025-11-13T08:07:47.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154062025-11-13T08:07:47.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
154072025-11-13T08:07:47.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
154082025-11-13T08:07:47.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.328013ms iteration = 36 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
154092025-11-13T08:07:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 54 reason = Timeout
154102025-11-13T08:07:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 107 reason = Timeout
154112025-11-13T08:07:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
154122025-11-13T08:07:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
154132025-11-13T08:07:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
154142025-11-13T08:07:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
154152025-11-13T08:07:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154162025-11-13T08:07:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154172025-11-13T08:07:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
154182025-11-13T08:07:52.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
154192025-11-13T08:07:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
154202025-11-13T08:07:52.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
154212025-11-13T08:07:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
154222025-11-13T08:07:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
154232025-11-13T08:07:52.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
154242025-11-13T08:07:52.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.399515ms iteration = 107 status = {"config_updated":false}
154252025-11-13T08:07:52.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
154262025-11-13T08:07:52.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
154272025-11-13T08:07:52.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154282025-11-13T08:07:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
154292025-11-13T08:07:52.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
154302025-11-13T08:07:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
154312025-11-13T08:07:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 54 reason = Timeout
154322025-11-13T08:07:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
154332025-11-13T08:07:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
154342025-11-13T08:07:52.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
154352025-11-13T08:07:52.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
154362025-11-13T08:07:52.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.883107ms iteration = 54 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
154372025-11-13T08:07:52.651ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
154382025-11-13T08:07:52.651ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.886939ms iteration = 54 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
154392025-11-13T08:07:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 446 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 2e4a8d7e-8777-4dae-9bb7-6e50d4a9de07 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
154402025-11-13T08:07:57.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 108 reason = Timeout
154412025-11-13T08:07:57.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
154422025-11-13T08:07:57.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
154432025-11-13T08:07:57.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154442025-11-13T08:07:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
154452025-11-13T08:07:57.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
154462025-11-13T08:07:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
154472025-11-13T08:07:57.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
154482025-11-13T08:07:57.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.643741ms iteration = 108 status = {"config_updated":false}
154492025-11-13T08:08:02.606ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 10 reason = Timeout
154502025-11-13T08:08:02.606ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 10 reason = Timeout
154512025-11-13T08:08:02.606ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 19 reason = Timeout
154522025-11-13T08:08:02.606ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
154532025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 19 reason = Timeout
154542025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 10 reason = Timeout
154552025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
154562025-11-13T08:08:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:58:02.606966363 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
154572025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
154582025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 12 reason = Timeout
154592025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 19 reason = Timeout
154602025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
154612025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 12 reason = Timeout
154622025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 10 reason = Timeout
154632025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 10 reason = Timeout
154642025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
154652025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
154662025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 10 reason = Timeout
154672025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
154682025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
154692025-11-13T08:08:02.607ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6490, tv_nsec: 959849799 } })
154702025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
154712025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
154722025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
154732025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
154742025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
154752025-11-13T08:08:02.607ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6490, tv_nsec: 960098074 } })
154762025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 11 reason = Timeout
154772025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
154782025-11-13T08:08:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
154792025-11-13T08:08:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
154802025-11-13T08:08:02.608ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = ExternalNetworking step id = Ensure
154812025-11-13T08:08:02.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
154822025-11-13T08:08:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
154832025-11-13T08:08:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.304425ms iteration = 10 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
154842025-11-13T08:08:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
154852025-11-13T08:08:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
154862025-11-13T08:08:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.287845ms iteration = 10 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
154872025-11-13T08:08:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
154882025-11-13T08:08:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
154892025-11-13T08:08:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
154902025-11-13T08:08:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
154912025-11-13T08:08:02.610ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154922025-11-13T08:08:02.611ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:58:02.606966363 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
154932025-11-13T08:08:02.611ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.251274ms iteration = 10 status = {"expiration":"2025-11-13T07:58:02.606966363Z","pruned":[]}
154942025-11-13T08:08:02.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154952025-11-13T08:08:02.612ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154962025-11-13T08:08:02.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
154972025-11-13T08:08:02.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
154982025-11-13T08:08:02.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
154992025-11-13T08:08:02.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
155002025-11-13T08:08:02.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
155012025-11-13T08:08:02.615ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6490, tv_nsec: 968290665 } })
155022025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 5 reason = Timeout
155032025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155042025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 10 reason = Timeout
155052025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 23 reason = Timeout
155062025-11-13T08:08:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
155072025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
155082025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155092025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155102025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155112025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
155122025-11-13T08:08:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155132025-11-13T08:08:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155142025-11-13T08:08:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155152025-11-13T08:08:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155162025-11-13T08:08:02.619ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6490, tv_nsec: 971740813 } })
155172025-11-13T08:08:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155182025-11-13T08:08:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155192025-11-13T08:08:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155202025-11-13T08:08:02.619ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
155212025-11-13T08:08:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
155222025-11-13T08:08:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155232025-11-13T08:08:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
155242025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155252025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
155262025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
155272025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
155282025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
155292025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
155302025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
155312025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
155322025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
155332025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
155342025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
155352025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
155362025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
155372025-11-13T08:08:02.622ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
155382025-11-13T08:08:02.623ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
155392025-11-13T08:08:02.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155402025-11-13T08:08:02.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 28 reason = Timeout
155412025-11-13T08:08:02.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
155422025-11-13T08:08:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
155432025-11-13T08:08:02.626ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155442025-11-13T08:08:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
155452025-11-13T08:08:02.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 30 reason = Timeout
155462025-11-13T08:08:02.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
155472025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155482025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155492025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155502025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155512025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155522025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 20.410501ms iteration = 19 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
155532025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 109 reason = Timeout
155542025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 55 reason = Timeout
155552025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
155562025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
155572025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
155582025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 47 reason = Timeout
155592025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
155602025-11-13T08:08:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
155612025-11-13T08:08:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
155622025-11-13T08:08:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
155632025-11-13T08:08:02.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155642025-11-13T08:08:02.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155652025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
155662025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
155672025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
155682025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
155692025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
155702025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
155712025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
155722025-11-13T08:08:02.633ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155732025-11-13T08:08:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155742025-11-13T08:08:02.634ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155752025-11-13T08:08:02.635ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
155762025-11-13T08:08:02.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
155772025-11-13T08:08:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
155782025-11-13T08:08:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
155792025-11-13T08:08:02.635ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
155802025-11-13T08:08:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
155812025-11-13T08:08:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
155822025-11-13T08:08:02.636ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
155832025-11-13T08:08:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
155842025-11-13T08:08:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
155852025-11-13T08:08:02.638ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
155862025-11-13T08:08:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
155872025-11-13T08:08:02.638ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
155882025-11-13T08:08:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
155892025-11-13T08:08:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
155902025-11-13T08:08:02.638ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155912025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 10 reason = Timeout
155922025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 19 reason = Timeout
155932025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
155942025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
155952025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
155962025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 19 reason = Timeout
155972025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
155982025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 10 reason = Timeout
155992025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
156002025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 19 reason = Timeout
156012025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
156022025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 28 reason = Timeout
156032025-11-13T08:08:02.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
156042025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 19 reason = Timeout
156052025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
156062025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
156072025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 19 reason = Timeout
156082025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
156092025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 19 reason = Timeout
156102025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
156112025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
156122025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
156132025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 55 reason = Timeout
156142025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
156152025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
156162025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
156172025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
156182025-11-13T08:08:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
156192025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
156202025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
156212025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
156222025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
156232025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
156242025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
156252025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
156262025-11-13T08:08:02.643ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156272025-11-13T08:08:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 10 reason = Timeout
156282025-11-13T08:08:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
156292025-11-13T08:08:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
156302025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 10 reason = Timeout
156312025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
156322025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 19 reason = Timeout
156332025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
156342025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 19 reason = Timeout
156352025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 37 reason = Timeout
156362025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
156372025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 37 reason = Timeout
156382025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
156392025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 19 reason = Timeout
156402025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
156412025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 19 reason = Timeout
156422025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
156432025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 19 reason = Timeout
156442025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 10 reason = Timeout
156452025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
156462025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
156472025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 19 reason = Timeout
156482025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
156492025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
156502025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
156512025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
156522025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 10 reason = Timeout
156532025-11-13T08:08:02.648ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
156542025-11-13T08:08:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
156552025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
156562025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
156572025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
156582025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
156592025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
156602025-11-13T08:08:02.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6491, tv_nsec: 1399856 } })
156612025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
156622025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
156632025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
156642025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
156652025-11-13T08:08:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
156662025-11-13T08:08:02.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156672025-11-13T08:08:02.654ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
156682025-11-13T08:08:02.654ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
156692025-11-13T08:08:02.654ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
156702025-11-13T08:08:02.654ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156712025-11-13T08:08:02.654ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156722025-11-13T08:08:02.655ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156732025-11-13T08:08:02.655ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156742025-11-13T08:08:02.655ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
156752025-11-13T08:08:02.655ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
156762025-11-13T08:08:02.655ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
156772025-11-13T08:08:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
156782025-11-13T08:08:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
156792025-11-13T08:08:02.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
156802025-11-13T08:08:02.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
156812025-11-13T08:08:02.658ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156822025-11-13T08:08:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
156832025-11-13T08:08:02.661ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
156842025-11-13T08:08:02.661ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
156852025-11-13T08:08:02.661ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
156862025-11-13T08:08:02.661ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
156872025-11-13T08:08:02.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
156882025-11-13T08:08:02.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
156892025-11-13T08:08:02.662ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
156902025-11-13T08:08:02.662ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
156912025-11-13T08:08:02.662ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
156922025-11-13T08:08:02.662ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
156932025-11-13T08:08:02.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
156942025-11-13T08:08:02.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
156952025-11-13T08:08:02.663ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
156962025-11-13T08:08:02.663ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
156972025-11-13T08:08:02.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
156982025-11-13T08:08:02.666ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156992025-11-13T08:08:02.666ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157002025-11-13T08:08:02.669ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 23.651985ms iteration = 10 status = {"errors":[],"found_port_ok":[]}
157012025-11-13T08:08:02.669ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
157022025-11-13T08:08:02.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
157032025-11-13T08:08:02.670ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
157042025-11-13T08:08:02.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
157052025-11-13T08:08:02.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
157062025-11-13T08:08:02.671ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
157072025-11-13T08:08:02.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
157082025-11-13T08:08:02.673ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
157092025-11-13T08:08:02.673ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 23.479431ms iteration = 10 status = {"by_rx":{},"error":null}
157102025-11-13T08:08:02.674ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
157112025-11-13T08:08:02.676ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
157122025-11-13T08:08:02.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157132025-11-13T08:08:02.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
157142025-11-13T08:08:02.676ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 28.065582ms iteration = 19 status = {"errors":[],"garbage_collect_requested":[]}
157152025-11-13T08:08:02.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157162025-11-13T08:08:02.677ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
157172025-11-13T08:08:02.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
157182025-11-13T08:08:02.679ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
157192025-11-13T08:08:02.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
157202025-11-13T08:08:02.685ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157212025-11-13T08:08:02.691ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 69.443346ms iteration = 5 status = {"nsuccess":5}
157222025-11-13T08:08:02.691ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157232025-11-13T08:08:02.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
157242025-11-13T08:08:02.697ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
157252025-11-13T08:08:02.697ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157262025-11-13T08:08:02.697ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157272025-11-13T08:08:02.697ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 49.002433ms iteration = 19 status = {"errors":[],"requests_created_ok":[]}
157282025-11-13T08:08:02.697ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157292025-11-13T08:08:02.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
157302025-11-13T08:08:02.698ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6491, tv_nsec: 50476471 } })
157312025-11-13T08:08:02.698ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
157322025-11-13T08:08:02.699ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
157332025-11-13T08:08:02.699ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
157342025-11-13T08:08:02.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
157352025-11-13T08:08:02.701ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 53.210756ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
157362025-11-13T08:08:02.701ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
157372025-11-13T08:08:02.701ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 74.23868ms iteration = 109 status = {"config_updated":false}
157382025-11-13T08:08:02.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
157392025-11-13T08:08:02.702ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
157402025-11-13T08:08:02.702ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
157412025-11-13T08:08:02.702ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
157422025-11-13T08:08:02.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
157432025-11-13T08:08:02.702ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
157442025-11-13T08:08:02.702ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
157452025-11-13T08:08:02.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
157462025-11-13T08:08:02.703ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 484d8991-1076-40d9-9a86-ae1c48054b42 original_time_started = 2025-11-13 08:07:04.309354 UTC
157472025-11-13T08:08:02.703ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 75.840971ms iteration = 47 status = {"Loaded":{"collection_id":"484d8991-1076-40d9-9a86-ae1c48054b42","time_loaded":"2025-11-13T08:08:02.627417165Z","time_started":"2025-11-13T08:07:04.309354Z"}}
157482025-11-13T08:08:02.703ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
157492025-11-13T08:08:02.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
157502025-11-13T08:08:02.704ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
157512025-11-13T08:08:02.704ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
157522025-11-13T08:08:02.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
157532025-11-13T08:08:02.705ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157542025-11-13T08:08:02.705ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 63.048279ms iteration = 55 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
157552025-11-13T08:08:02.705ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
157562025-11-13T08:08:02.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
157572025-11-13T08:08:02.707ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
157582025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
157592025-11-13T08:08:02.713ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
157602025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
157612025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
157622025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
157632025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 71.161189ms iteration = 19 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
157642025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 106.506405ms iteration = 10 status = {"generation":2}
157652025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
157662025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
157672025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
157682025-11-13T08:08:02.713ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 106.715608ms iteration = 10 status = {"generation":1}
157692025-11-13T08:08:02.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
157702025-11-13T08:08:02.717ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157712025-11-13T08:08:02.718ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
157722025-11-13T08:08:02.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
157732025-11-13T08:08:02.719ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 77.253189ms iteration = 19 status = 0
157742025-11-13T08:08:02.719ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
157752025-11-13T08:08:02.720ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
157762025-11-13T08:08:02.721ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 72.76619ms iteration = 10 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
157772025-11-13T08:08:02.724ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
157782025-11-13T08:08:02.724ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
157792025-11-13T08:08:02.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
157802025-11-13T08:08:02.731ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157812025-11-13T08:08:02.734ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
157822025-11-13T08:08:02.734ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
157832025-11-13T08:08:02.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
157842025-11-13T08:08:02.735ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
157852025-11-13T08:08:02.735ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 87.303757ms iteration = 37 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
157862025-11-13T08:08:02.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157872025-11-13T08:08:02.740ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
157882025-11-13T08:08:02.743ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157892025-11-13T08:08:02.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157902025-11-13T08:08:02.745ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157912025-11-13T08:08:02.745ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157922025-11-13T08:08:02.747ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
157932025-11-13T08:08:02.748ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 100.1499ms iteration = 19 status = {"errors":[],"finish_invoked_ok":[]}
157942025-11-13T08:08:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
157952025-11-13T08:08:02.749ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
157962025-11-13T08:08:02.749ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
157972025-11-13T08:08:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
157982025-11-13T08:08:02.751ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
157992025-11-13T08:08:02.751ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
158002025-11-13T08:08:02.751ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
158012025-11-13T08:08:02.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158022025-11-13T08:08:02.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
158032025-11-13T08:08:02.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158042025-11-13T08:08:02.752ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
158052025-11-13T08:08:02.752ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
158062025-11-13T08:08:02.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
158072025-11-13T08:08:02.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158082025-11-13T08:08:02.757ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
158092025-11-13T08:08:02.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158102025-11-13T08:08:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158112025-11-13T08:08:02.762ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
158122025-11-13T08:08:02.768ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
158132025-11-13T08:08:02.772ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
158142025-11-13T08:08:02.772ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
158152025-11-13T08:08:02.772ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 154.185081ms iteration = 10 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
158162025-11-13T08:08:02.773ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
158172025-11-13T08:08:02.775ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
158182025-11-13T08:08:02.777ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
158192025-11-13T08:08:02.777ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
158202025-11-13T08:08:02.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
158212025-11-13T08:08:02.778ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158222025-11-13T08:08:02.779ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
158232025-11-13T08:08:02.781ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
158242025-11-13T08:08:02.781ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158252025-11-13T08:08:02.781ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
158262025-11-13T08:08:02.781ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
158272025-11-13T08:08:02.781ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 139.645805ms iteration = 10 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
158282025-11-13T08:08:02.782ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
158292025-11-13T08:08:02.782ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
158302025-11-13T08:08:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
158312025-11-13T08:08:02.782ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
158322025-11-13T08:08:02.782ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
158332025-11-13T08:08:02.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
158342025-11-13T08:08:02.784ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
158352025-11-13T08:08:02.785ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
158362025-11-13T08:08:02.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
158372025-11-13T08:08:02.785ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
158382025-11-13T08:08:02.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
158392025-11-13T08:08:02.786ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = SledList step id = Fetch
158402025-11-13T08:08:02.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158412025-11-13T08:08:02.786ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158422025-11-13T08:08:02.786ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 138.079744ms iteration = 37 status = "NoSitrep"
158432025-11-13T08:08:02.786ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
158442025-11-13T08:08:02.786ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
158452025-11-13T08:08:02.786ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
158462025-11-13T08:08:02.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158472025-11-13T08:08:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
158482025-11-13T08:08:02.790ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
158492025-11-13T08:08:02.790ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
158502025-11-13T08:08:02.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
158512025-11-13T08:08:02.791ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
158522025-11-13T08:08:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
158532025-11-13T08:08:02.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158542025-11-13T08:08:02.793ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
158552025-11-13T08:08:02.793ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 166.917882ms iteration = 28 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
158562025-11-13T08:08:02.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
158572025-11-13T08:08:02.796ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158582025-11-13T08:08:02.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158592025-11-13T08:08:02.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158602025-11-13T08:08:02.800ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
158612025-11-13T08:08:02.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
158622025-11-13T08:08:02.803ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
158632025-11-13T08:08:02.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158642025-11-13T08:08:02.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158652025-11-13T08:08:02.812ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = DeployNexusRecords step id = Ensure
158662025-11-13T08:08:02.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158672025-11-13T08:08:02.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158682025-11-13T08:08:02.816ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158692025-11-13T08:08:02.819ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
158702025-11-13T08:08:02.819ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158712025-11-13T08:08:02.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158722025-11-13T08:08:02.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
158732025-11-13T08:08:02.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
158742025-11-13T08:08:02.828ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
158752025-11-13T08:08:02.828ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 201.049503ms iteration = 55 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
158762025-11-13T08:08:02.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158772025-11-13T08:08:02.841ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
158782025-11-13T08:08:02.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158792025-11-13T08:08:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158802025-11-13T08:08:02.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158812025-11-13T08:08:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158822025-11-13T08:08:02.862ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158832025-11-13T08:08:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158842025-11-13T08:08:02.868ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158852025-11-13T08:08:02.868ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158862025-11-13T08:08:02.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158872025-11-13T08:08:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158882025-11-13T08:08:02.871ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 244.67866ms iteration = 30 status = {"physical_disks_added":0}
158892025-11-13T08:08:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158902025-11-13T08:08:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
158912025-11-13T08:08:02.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158922025-11-13T08:08:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158932025-11-13T08:08:02.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158942025-11-13T08:08:02.886ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
158952025-11-13T08:08:02.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158962025-11-13T08:08:02.886ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
158972025-11-13T08:08:02.888ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 246.140309ms iteration = 19 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
158982025-11-13T08:08:02.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
158992025-11-13T08:08:02.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159002025-11-13T08:08:02.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159012025-11-13T08:08:02.904ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 255.584045ms iteration = 19 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
159022025-11-13T08:08:02.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
159032025-11-13T08:08:02.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159042025-11-13T08:08:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159052025-11-13T08:08:02.906ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
159062025-11-13T08:08:02.906ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
159072025-11-13T08:08:02.906ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
159082025-11-13T08:08:02.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
159092025-11-13T08:08:02.909ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159102025-11-13T08:08:02.909ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159112025-11-13T08:08:02.915ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
159122025-11-13T08:08:02.915ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 309.024565ms iteration = 10 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
159132025-11-13T08:08:02.919ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159142025-11-13T08:08:02.921ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
159152025-11-13T08:08:02.922ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "96adc4ce-f02e-4d00-80b8-fd7be9789504", "content-length": "78", "date": "Thu, 13 Nov 2025 08:08:02 GMT"} })
159162025-11-13T08:08:02.922ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
159172025-11-13T08:08:02.922ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
159182025-11-13T08:08:02.922ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
159192025-11-13T08:08:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
159202025-11-13T08:08:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
159212025-11-13T08:08:02.922ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
159222025-11-13T08:08:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
159232025-11-13T08:08:02.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159242025-11-13T08:08:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159252025-11-13T08:08:02.925ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
159262025-11-13T08:08:02.926ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
159272025-11-13T08:08:02.926ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
159282025-11-13T08:08:02.927ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
159292025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
159302025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159312025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
159322025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
159332025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
159342025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159352025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
159362025-11-13T08:08:02.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
159372025-11-13T08:08:02.932ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
159382025-11-13T08:08:02.932ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
159392025-11-13T08:08:02.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
159402025-11-13T08:08:02.933ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159412025-11-13T08:08:02.934ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
159422025-11-13T08:08:02.934ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
159432025-11-13T08:08:02.934ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "70828dcc-3e91-4e57-9b3e-79e91b05e5dd", "content-length": "81", "date": "Thu, 13 Nov 2025 08:08:02 GMT"} })
159442025-11-13T08:08:02.937ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = SledAgent step id = Ensure
159452025-11-13T08:08:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
159462025-11-13T08:08:02.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159472025-11-13T08:08:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159482025-11-13T08:08:02.938ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
159492025-11-13T08:08:02.940ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
159502025-11-13T08:08:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
159512025-11-13T08:08:02.957ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
159522025-11-13T08:08:02.957ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
159532025-11-13T08:08:02.958ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
159542025-11-13T08:08:02.958ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffd2737c-4301-4d1d-820f-935baff9048a", "content-length": "26", "date": "Thu, 13 Nov 2025 08:08:02 GMT"} })
159552025-11-13T08:08:02.958ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
159562025-11-13T08:08:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
159572025-11-13T08:08:02.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159582025-11-13T08:08:02.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159592025-11-13T08:08:02.960ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
159602025-11-13T08:08:02.961ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
159612025-11-13T08:08:02.962ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 355.147432ms iteration = 19 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
159622025-11-13T08:08:02.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159632025-11-13T08:08:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159642025-11-13T08:08:02.963ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
159652025-11-13T08:08:02.963ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
159662025-11-13T08:08:02.964ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e1cc84c-1a5c-415e-8a06-4e6ec97b85d7", "content-length": "26", "date": "Thu, 13 Nov 2025 08:08:02 GMT"} })
159672025-11-13T08:08:02.964ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
159682025-11-13T08:08:02.964ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159692025-11-13T08:08:02.966ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
159702025-11-13T08:08:02.969ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
159712025-11-13T08:08:02.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159722025-11-13T08:08:02.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159732025-11-13T08:08:02.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159742025-11-13T08:08:02.981ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
159752025-11-13T08:08:02.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159762025-11-13T08:08:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159772025-11-13T08:08:02.986ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159782025-11-13T08:08:02.986ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159792025-11-13T08:08:02.989ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
159802025-11-13T08:08:02.989ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
159812025-11-13T08:08:02.989ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 347.429701ms iteration = 19 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
159822025-11-13T08:08:02.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159832025-11-13T08:08:02.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159842025-11-13T08:08:02.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159852025-11-13T08:08:02.996ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
159862025-11-13T08:08:02.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159872025-11-13T08:08:02.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159882025-11-13T08:08:03.000ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
159892025-11-13T08:08:03.000ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
159902025-11-13T08:08:03.000ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "453b425c-ee9e-4291-94c3-b2ae82eace65", "content-length": "81", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
159912025-11-13T08:08:03.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159922025-11-13T08:08:03.005ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
159932025-11-13T08:08:03.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159942025-11-13T08:08:03.005ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159952025-11-13T08:08:03.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159962025-11-13T08:08:03.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159972025-11-13T08:08:03.013ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
159982025-11-13T08:08:03.014ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
159992025-11-13T08:08:03.016ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
160002025-11-13T08:08:03.016ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
160012025-11-13T08:08:03.016ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd9c6408-bf28-41f0-a04c-2c713f0d243e", "content-length": "81", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
160022025-11-13T08:08:03.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160032025-11-13T08:08:03.019ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 371.204337ms iteration = 19 status = {"disabled":false,"errors":[],"sps":[]}
160042025-11-13T08:08:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
160052025-11-13T08:08:03.032ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
160062025-11-13T08:08:03.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
160072025-11-13T08:08:03.034ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160082025-11-13T08:08:03.038ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
160092025-11-13T08:08:03.038ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
160102025-11-13T08:08:03.038ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
160112025-11-13T08:08:03.040ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160122025-11-13T08:08:03.043ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
160132025-11-13T08:08:03.043ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
160142025-11-13T08:08:03.043ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
160152025-11-13T08:08:03.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160162025-11-13T08:08:03.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160172025-11-13T08:08:03.072ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
160182025-11-13T08:08:03.072ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
160192025-11-13T08:08:03.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160202025-11-13T08:08:03.073ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "261a523e-bb3f-4286-a9eb-c230683b3fbb", "content-length": "372", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
160212025-11-13T08:08:03.073ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160222025-11-13T08:08:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160232025-11-13T08:08:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160242025-11-13T08:08:03.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160252025-11-13T08:08:03.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
160262025-11-13T08:08:03.096ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
160272025-11-13T08:08:03.096ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "524e1a02-9019-442c-99aa-28a8a6a6557e", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
160282025-11-13T08:08:03.096ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
160292025-11-13T08:08:03.097ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
160302025-11-13T08:08:03.097ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
160312025-11-13T08:08:03.097ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "a15183da-4c89-4628-abaf-3f3a2455f3e9", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
160322025-11-13T08:08:03.097ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160332025-11-13T08:08:03.098ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160342025-11-13T08:08:03.098ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
160352025-11-13T08:08:03.098ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 449.852673ms iteration = 10 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
160362025-11-13T08:08:03.101ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160372025-11-13T08:08:03.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160382025-11-13T08:08:03.105ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160392025-11-13T08:08:03.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160402025-11-13T08:08:03.109ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
160412025-11-13T08:08:03.110ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
160422025-11-13T08:08:03.110ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ac922300-bb14-4e5c-88f3-ff2b215c82e7", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
160432025-11-13T08:08:03.113ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 470.661093ms iteration = 28 status = {}
160442025-11-13T08:08:03.113ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 29 reason = Dependency
160452025-11-13T08:08:03.113ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
160462025-11-13T08:08:03.113ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160472025-11-13T08:08:03.113ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160482025-11-13T08:08:03.115ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160492025-11-13T08:08:03.117ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
160502025-11-13T08:08:03.117ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160512025-11-13T08:08:03.117ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160522025-11-13T08:08:03.119ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160532025-11-13T08:08:03.121ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
160542025-11-13T08:08:03.121ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
160552025-11-13T08:08:03.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
160562025-11-13T08:08:03.128ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160572025-11-13T08:08:03.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160582025-11-13T08:08:03.131ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160592025-11-13T08:08:03.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160602025-11-13T08:08:03.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160612025-11-13T08:08:03.137ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
160622025-11-13T08:08:03.137ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160632025-11-13T08:08:03.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160642025-11-13T08:08:03.139ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160652025-11-13T08:08:03.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160662025-11-13T08:08:03.141ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160672025-11-13T08:08:03.144ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
160682025-11-13T08:08:03.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160692025-11-13T08:08:03.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160702025-11-13T08:08:03.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160712025-11-13T08:08:03.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160722025-11-13T08:08:03.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160732025-11-13T08:08:03.150ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
160742025-11-13T08:08:03.150ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160752025-11-13T08:08:03.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160762025-11-13T08:08:03.152ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160772025-11-13T08:08:03.152ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
160782025-11-13T08:08:03.152ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 510.217121ms iteration = 10 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
160792025-11-13T08:08:03.152ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160802025-11-13T08:08:03.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160812025-11-13T08:08:03.157ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
160822025-11-13T08:08:03.157ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
160832025-11-13T08:08:03.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
160842025-11-13T08:08:03.175ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
160852025-11-13T08:08:03.177ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "f45f8216-3095-4708-b6c3-4b64b1d96531", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
160862025-11-13T08:08:03.179ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = FirewallRules step id = Ensure
160872025-11-13T08:08:03.179ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160882025-11-13T08:08:03.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160892025-11-13T08:08:03.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160902025-11-13T08:08:03.181ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
160912025-11-13T08:08:03.181ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160922025-11-13T08:08:03.183ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160932025-11-13T08:08:03.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160942025-11-13T08:08:03.185ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160952025-11-13T08:08:03.186ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160962025-11-13T08:08:03.188ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160972025-11-13T08:08:03.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
160982025-11-13T08:08:03.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
160992025-11-13T08:08:03.192ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161002025-11-13T08:08:03.193ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161012025-11-13T08:08:03.199ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
161022025-11-13T08:08:03.199ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161032025-11-13T08:08:03.200ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161042025-11-13T08:08:03.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161052025-11-13T08:08:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161062025-11-13T08:08:03.203ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
161072025-11-13T08:08:03.203ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
161082025-11-13T08:08:03.203ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = f159f7f0-50d4-4554-90f3-1594d4e406f2 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
161092025-11-13T08:08:03.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
161102025-11-13T08:08:03.203ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
161112025-11-13T08:08:03.203ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161122025-11-13T08:08:03.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161132025-11-13T08:08:03.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161142025-11-13T08:08:03.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161152025-11-13T08:08:03.210ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
161162025-11-13T08:08:03.210ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
161172025-11-13T08:08:03.211ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
161182025-11-13T08:08:03.211ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
161192025-11-13T08:08:03.211ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "2fec96ee-1c9a-4386-a063-538e05d09038", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
161202025-11-13T08:08:03.211ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
161212025-11-13T08:08:03.212ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
161222025-11-13T08:08:03.212ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161232025-11-13T08:08:03.212ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161242025-11-13T08:08:03.214ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "afebab32-b495-48f5-b11f-0628032a4c9a", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
161252025-11-13T08:08:03.214ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
161262025-11-13T08:08:03.215ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
161272025-11-13T08:08:03.215ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
161282025-11-13T08:08:03.215ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 597.513327ms iteration = 23 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
161292025-11-13T08:08:03.215ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161302025-11-13T08:08:03.216ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
161312025-11-13T08:08:03.216ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 609.063684ms iteration = 12 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
161322025-11-13T08:08:03.216ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161332025-11-13T08:08:03.217ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161342025-11-13T08:08:03.218ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161352025-11-13T08:08:03.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161362025-11-13T08:08:03.220ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161372025-11-13T08:08:03.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161382025-11-13T08:08:03.222ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161392025-11-13T08:08:03.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161402025-11-13T08:08:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161412025-11-13T08:08:03.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161422025-11-13T08:08:03.225ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161432025-11-13T08:08:03.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161442025-11-13T08:08:03.231ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
161452025-11-13T08:08:03.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161462025-11-13T08:08:03.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161472025-11-13T08:08:03.232ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161482025-11-13T08:08:03.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161492025-11-13T08:08:03.234ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161502025-11-13T08:08:03.234ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161512025-11-13T08:08:03.237ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161522025-11-13T08:08:03.237ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161532025-11-13T08:08:03.239ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
161542025-11-13T08:08:03.239ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161552025-11-13T08:08:03.239ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "623e8296-790a-448a-afdb-2c036cefba1a", "content-length": "691", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
161562025-11-13T08:08:03.239ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
161572025-11-13T08:08:03.239ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
161582025-11-13T08:08:03.239ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
161592025-11-13T08:08:03.239ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161602025-11-13T08:08:03.241ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161612025-11-13T08:08:03.241ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161622025-11-13T08:08:03.242ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 599.941254ms iteration = 19 status = {}
161632025-11-13T08:08:03.243ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161642025-11-13T08:08:03.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161652025-11-13T08:08:03.244ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161662025-11-13T08:08:03.247ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 640.467311ms iteration = 19 status = {}
161672025-11-13T08:08:03.249ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
161682025-11-13T08:08:03.249ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161692025-11-13T08:08:03.249ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161702025-11-13T08:08:03.250ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161712025-11-13T08:08:03.251ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161722025-11-13T08:08:03.252ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161732025-11-13T08:08:03.252ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161742025-11-13T08:08:03.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161752025-11-13T08:08:03.259ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
161762025-11-13T08:08:03.260ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161772025-11-13T08:08:03.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161782025-11-13T08:08:03.260ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6491, tv_nsec: 612982240 } })
161792025-11-13T08:08:03.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161802025-11-13T08:08:03.264ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161812025-11-13T08:08:03.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161822025-11-13T08:08:03.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161832025-11-13T08:08:03.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161842025-11-13T08:08:03.267ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161852025-11-13T08:08:03.269ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161862025-11-13T08:08:03.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161872025-11-13T08:08:03.271ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161882025-11-13T08:08:03.271ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161892025-11-13T08:08:03.272ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161902025-11-13T08:08:03.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161912025-11-13T08:08:03.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161922025-11-13T08:08:03.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161932025-11-13T08:08:03.275ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161942025-11-13T08:08:03.280ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
161952025-11-13T08:08:03.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161962025-11-13T08:08:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161972025-11-13T08:08:03.281ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
161982025-11-13T08:08:03.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
161992025-11-13T08:08:03.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162002025-11-13T08:08:03.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162012025-11-13T08:08:03.284ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162022025-11-13T08:08:03.289ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
162032025-11-13T08:08:03.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162042025-11-13T08:08:03.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162052025-11-13T08:08:03.289ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
162062025-11-13T08:08:03.290ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "3c1d819b-8cc2-41a9-b527-00c873c9b386", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
162072025-11-13T08:08:03.290ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
162082025-11-13T08:08:03.292ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "75eb1137-95ae-4f57-b331-deb95289a82f", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
162092025-11-13T08:08:03.292ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
162102025-11-13T08:08:03.294ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
162112025-11-13T08:08:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162122025-11-13T08:08:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162132025-11-13T08:08:03.295ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "d358a82f-75a9-48dd-8358-b270d90a4c1c", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
162142025-11-13T08:08:03.295ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
162152025-11-13T08:08:03.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162162025-11-13T08:08:03.297ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
162172025-11-13T08:08:03.297ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 690.180699ms iteration = 12 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
162182025-11-13T08:08:03.297ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
162192025-11-13T08:08:03.297ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162202025-11-13T08:08:03.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162212025-11-13T08:08:03.299ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
162222025-11-13T08:08:03.299ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
162232025-11-13T08:08:03.299ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
162242025-11-13T08:08:03.333ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
162252025-11-13T08:08:03.333ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
162262025-11-13T08:08:03.334ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "92e69770-5bf2-4b2e-abaa-dba8b9f0fa29", "content-length": "372", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
162272025-11-13T08:08:03.334ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
162282025-11-13T08:08:03.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
162292025-11-13T08:08:03.350ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
162302025-11-13T08:08:03.350ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
162312025-11-13T08:08:03.351ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "d6c4a02b-9989-4ed3-80b3-a10571dda0f5", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
162322025-11-13T08:08:03.351ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
162332025-11-13T08:08:03.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
162342025-11-13T08:08:03.354ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
162352025-11-13T08:08:03.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
162362025-11-13T08:08:03.357ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
162372025-11-13T08:08:03.357ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
162382025-11-13T08:08:03.360ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
162392025-11-13T08:08:03.360ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
162402025-11-13T08:08:03.362ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d0d5e4a5-c517-4248-b4e4-7847c374eee0", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
162412025-11-13T08:08:03.363ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 250.794561ms iteration = 29 status = {}
162422025-11-13T08:08:03.366ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
162432025-11-13T08:08:03.366ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
162442025-11-13T08:08:03.391ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162452025-11-13T08:08:03.391ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
162462025-11-13T08:08:03.393ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
162472025-11-13T08:08:03.394ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162482025-11-13T08:08:03.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
162492025-11-13T08:08:03.402ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162502025-11-13T08:08:03.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162512025-11-13T08:08:03.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162522025-11-13T08:08:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162532025-11-13T08:08:03.406ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162542025-11-13T08:08:03.406ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
162552025-11-13T08:08:03.406ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
162562025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
162572025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
162582025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
162592025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
162602025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
162612025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
162622025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
162632025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
162642025-11-13T08:08:03.407ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
162652025-11-13T08:08:03.407ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
162662025-11-13T08:08:03.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
162672025-11-13T08:08:03.410ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
162682025-11-13T08:08:03.410ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
162692025-11-13T08:08:03.411ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
162702025-11-13T08:08:03.411ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
162712025-11-13T08:08:03.411ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "cc6966d8-ef68-4d2b-bada-436a1d60dc1b", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
162722025-11-13T08:08:03.413ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
162732025-11-13T08:08:03.413ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = Dns step id = Ensure
162742025-11-13T08:08:03.413ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
162752025-11-13T08:08:03.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162762025-11-13T08:08:03.413ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162772025-11-13T08:08:03.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162782025-11-13T08:08:03.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162792025-11-13T08:08:03.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162802025-11-13T08:08:03.419ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
162812025-11-13T08:08:03.420ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
162822025-11-13T08:08:03.420ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
162832025-11-13T08:08:03.420ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
162842025-11-13T08:08:03.420ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162852025-11-13T08:08:03.420ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162862025-11-13T08:08:03.421ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162872025-11-13T08:08:03.424ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
162882025-11-13T08:08:03.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162892025-11-13T08:08:03.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162902025-11-13T08:08:03.426ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
162912025-11-13T08:08:03.426ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
162922025-11-13T08:08:03.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162932025-11-13T08:08:03.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162942025-11-13T08:08:03.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162952025-11-13T08:08:03.431ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
162962025-11-13T08:08:03.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
162972025-11-13T08:08:03.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
162982025-11-13T08:08:03.433ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
162992025-11-13T08:08:03.433ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
163002025-11-13T08:08:03.433ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
163012025-11-13T08:08:03.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163022025-11-13T08:08:03.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163032025-11-13T08:08:03.434ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163042025-11-13T08:08:03.434ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163052025-11-13T08:08:03.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163062025-11-13T08:08:03.438ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
163072025-11-13T08:08:03.481ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
163082025-11-13T08:08:03.481ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
163092025-11-13T08:08:03.481ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
163102025-11-13T08:08:03.481ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163112025-11-13T08:08:03.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163122025-11-13T08:08:03.484ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163132025-11-13T08:08:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
163142025-11-13T08:08:03.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163152025-11-13T08:08:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163162025-11-13T08:08:03.490ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
163172025-11-13T08:08:03.490ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
163182025-11-13T08:08:03.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163192025-11-13T08:08:03.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163202025-11-13T08:08:03.492ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163212025-11-13T08:08:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
163222025-11-13T08:08:03.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163232025-11-13T08:08:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163242025-11-13T08:08:03.498ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
163252025-11-13T08:08:03.498ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
163262025-11-13T08:08:03.498ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163272025-11-13T08:08:03.498ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163282025-11-13T08:08:03.499ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163292025-11-13T08:08:03.501ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
163302025-11-13T08:08:03.501ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163312025-11-13T08:08:03.502ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163322025-11-13T08:08:03.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163332025-11-13T08:08:03.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163342025-11-13T08:08:03.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163352025-11-13T08:08:03.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163362025-11-13T08:08:03.506ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163372025-11-13T08:08:03.509ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
163382025-11-13T08:08:03.511ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
163392025-11-13T08:08:03.511ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
163402025-11-13T08:08:03.511ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = OmicronZones step id = Cleanup
163412025-11-13T08:08:03.511ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = OmicronZones step id = Cleanup
163422025-11-13T08:08:03.511ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = PhysicalDisks step id = Cleanup
163432025-11-13T08:08:03.511ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = Clickhouse step id = Ensure
163442025-11-13T08:08:03.511ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = Clickhouse step id = Ensure
163452025-11-13T08:08:03.542ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
163462025-11-13T08:08:03.547ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "eaf51aad-676d-421b-89c0-367b5359a834", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
163472025-11-13T08:08:03.548ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = SupportBundles step id = Cleanup
163482025-11-13T08:08:03.548ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163492025-11-13T08:08:03.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163502025-11-13T08:08:03.550ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163512025-11-13T08:08:03.551ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
163522025-11-13T08:08:03.551ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163532025-11-13T08:08:03.552ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163542025-11-13T08:08:03.553ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163552025-11-13T08:08:03.553ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163562025-11-13T08:08:03.555ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163572025-11-13T08:08:03.558ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
163582025-11-13T08:08:03.597ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = OmicronZones step id = Cleanup
163592025-11-13T08:08:03.597ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
163602025-11-13T08:08:03.597ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
163612025-11-13T08:08:03.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163622025-11-13T08:08:03.597ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163632025-11-13T08:08:03.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163642025-11-13T08:08:03.600ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
163652025-11-13T08:08:03.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163662025-11-13T08:08:03.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163672025-11-13T08:08:03.642ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
163682025-11-13T08:08:03.642ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
163692025-11-13T08:08:03.642ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = Cockroach step id = Ensure
163702025-11-13T08:08:03.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
163712025-11-13T08:08:03.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
163722025-11-13T08:08:03.652ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "82e2e5d3-ef7b-4428-829c-352c4a2f16be", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
163732025-11-13T08:08:03.652ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
163742025-11-13T08:08:03.652ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
163752025-11-13T08:08:03.653ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9191c2c-7ca2-4dff-8451-d018953efe07", "content-length": "682", "date": "Thu, 13 Nov 2025 08:08:03 GMT"} })
163762025-11-13T08:08:03.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
163772025-11-13T08:08:03.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
163782025-11-13T08:08:03.657ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
163792025-11-13T08:08:03.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
163802025-11-13T08:08:03.658ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
163812025-11-13T08:08:03.658ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 4262437e-53cc-446f-a1e0-2db6bf74f746 step component = MgsUpdates step id = Ensure
163822025-11-13T08:08:03.658ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Dependency
163832025-11-13T08:08:03.658ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 29 reason = Dependency
163842025-11-13T08:08:03.658ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163852025-11-13T08:08:03.658ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
163862025-11-13T08:08:03.659ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.051354709s iteration = 11 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3520,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":178486889,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":178486839,"secs":0}},"event_index":1,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":178500759,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":26605223,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":26605183,"secs":0}},"event_index":2,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":205124213,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":124839444,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":124839395,"secs":0}},"event_index":3,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":329990177,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":241507058,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":241506998,"secs":0}},"event_index":4,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":571511856,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":233916819,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":233916769,"secs":0}},"event_index":5,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":805443405,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":98038527,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":98038497,"secs":0}},"event_index":6,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":903490242,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3961,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3931,"secs":0}},"event_index":7,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":903505483,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2410,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2370,"secs":0}},"event_index":8,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":903510023,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2980,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":2940,"secs":0}},"event_index":9,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":903514853,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2150,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2110,"secs":0}},"event_index":10,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":903518813,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":37457566,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":37457526,"secs":0}},"event_index":11,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":940977799,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":48594306,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":48594276,"secs":0}},"event_index":12,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":989582215,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":45395953,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":45395923,"secs":0}},"event_index":13,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":34994378,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":15971264,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":15971214,"secs":0}},"event_index":14,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":50983122,"secs":1}},{"data":{"attempt_elapsed":{"nanos":6150,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":6110,"secs":0}},"event_index":15,"execution_id":"b3bbc3b5-82e6-4e70-9e63-419ec77ef80d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":51001222,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
163872025-11-13T08:08:03.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
163882025-11-13T08:08:03.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
163892025-11-13T08:08:03.661ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163902025-11-13T08:08:03.661ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163912025-11-13T08:08:03.661ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
163922025-11-13T08:08:03.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
163932025-11-13T08:08:03.663ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
163942025-11-13T08:08:03.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
163952025-11-13T08:08:03.664ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
163962025-11-13T08:08:03.664ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163972025-11-13T08:08:03.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
163982025-11-13T08:08:03.664ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
163992025-11-13T08:08:03.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
164002025-11-13T08:08:03.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
164012025-11-13T08:08:03.671ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.028644533s iteration = 19 status = {}
164022025-11-13T08:08:03.699ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
164032025-11-13T08:08:03.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
164042025-11-13T08:08:03.701ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164052025-11-13T08:08:03.704ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
164062025-11-13T08:08:03.704ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
164072025-11-13T08:08:03.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
164082025-11-13T08:08:03.746ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
164092025-11-13T08:08:03.746ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 87.50184ms iteration = 29 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
164102025-11-13T08:08:03.849ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(db0127fd-bd52-4625-be88-6185cc000069, 0), (4ba91f9a-a9b4-4c47-93bd-d51c608704f2, 0), (777e6448-5e33-4eb2-b0e2-7fa418592ff2, 0), (5588c052-6347-48b0-b6e1-745a625f895b, 0)] collection_id = db0127fd-bd52-4625-be88-6185cc000069
164112025-11-13T08:08:03.849ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
164122025-11-13T08:08:03.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
164132025-11-13T08:08:04.061ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = db0127fd-bd52-4625-be88-6185cc000069 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
164142025-11-13T08:08:04.061ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
164152025-11-13T08:08:04.061ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
164162025-11-13T08:08:04.141ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(777e6448-5e33-4eb2-b0e2-7fa418592ff2, 0), (5588c052-6347-48b0-b6e1-745a625f895b, 0), (484d8991-1076-40d9-9a86-ae1c48054b42, 0)]
164172025-11-13T08:08:04.142ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6492, tv_nsec: 494499200 } })
164182025-11-13T08:08:04.174ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6492, tv_nsec: 527117101 } })
164192025-11-13T08:08:04.175ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
164202025-11-13T08:08:04.175ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
164212025-11-13T08:08:04.175ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
164222025-11-13T08:08:04.175ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "50d95dbf-3bbb-4746-b5e4-9145c2c3f30f", "content-length": "476", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164232025-11-13T08:08:04.176ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
164242025-11-13T08:08:04.176ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bbc98a3-57a3-42d9-bf70-b27308356c17", "content-length": "734", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164252025-11-13T08:08:04.176ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
164262025-11-13T08:08:04.176ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6097175-527b-4216-80d0-d2a31fcaeb50", "content-length": "111", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164272025-11-13T08:08:04.176ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
164282025-11-13T08:08:04.177ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c434266-179d-4587-9ca5-80eff9b64470", "content-length": "111", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164292025-11-13T08:08:04.177ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
164302025-11-13T08:08:04.177ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d44fac9-375a-4f3d-90c5-c7f4be03c9c1", "content-length": "163", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164312025-11-13T08:08:04.177ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
164322025-11-13T08:08:04.177ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "233b21d3-4acd-47bd-a67d-6d1711d50518", "content-length": "163", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164332025-11-13T08:08:04.177ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
164342025-11-13T08:08:04.178ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "982b47cb-fb73-4c08-8834-c0f39b9b0565", "content-length": "166", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164352025-11-13T08:08:04.178ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
164362025-11-13T08:08:04.178ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66af029d-bb5d-47c3-ae4e-47b7673889f4", "content-length": "166", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164372025-11-13T08:08:04.178ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
164382025-11-13T08:08:04.178ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75311374-b261-439f-be85-eb93c6023dfe", "content-length": "702", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164392025-11-13T08:08:04.178ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
164402025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "860fe7f9-8b87-4ffc-9136-ebfc227fc947", "content-length": "727", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164412025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
164422025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a24dd2e4-574d-4179-a45c-ea836c0c752e", "content-length": "729", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164432025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
164442025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c51f331-07bc-4c5d-a69f-2c7576cd2133", "content-length": "728", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164452025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
164462025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad771ccc-77b4-45a4-bbbb-928804d4784a", "content-length": "722", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164472025-11-13T08:08:04.179ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
164482025-11-13T08:08:04.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb34dfd1-8095-40d8-b6a2-27980a4dc247", "content-length": "10", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164492025-11-13T08:08:04.180ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
164502025-11-13T08:08:04.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d10a0a6-12d7-4786-a062-2b19f2d17261", "content-length": "144", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164512025-11-13T08:08:04.180ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
164522025-11-13T08:08:04.180ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ced087de-2fd9-4382-9548-9e8b256d5d92", "content-length": "147", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164532025-11-13T08:08:04.180ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
164542025-11-13T08:08:04.181ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5701c3f6-492f-466a-9bd0-af251295c80d", "content-length": "109", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164552025-11-13T08:08:04.181ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
164562025-11-13T08:08:04.181ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "264963ef-f1e0-4884-a424-129387d8b12f", "content-length": "109", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164572025-11-13T08:08:04.181ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
164582025-11-13T08:08:04.181ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92cc78ba-89ea-41d8-8af5-c94d826de6dd", "content-length": "163", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164592025-11-13T08:08:04.181ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
164602025-11-13T08:08:04.182ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5be0c17-ba7c-49b9-b4d0-2b6911b4bee4", "content-length": "163", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164612025-11-13T08:08:04.182ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
164622025-11-13T08:08:04.182ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fb7b7e6-e0b0-4ea3-b289-b329326a1245", "content-length": "166", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164632025-11-13T08:08:04.182ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
164642025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7056f72b-eefe-4551-81f7-b18e629c306f", "content-length": "166", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164652025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
164662025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "796922ce-31fe-4256-830e-e1b4a45a88e4", "content-length": "702", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164672025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
164682025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fd1fa6b-1a8a-409f-a2f9-fb97f2651bc7", "content-length": "727", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164692025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
164702025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b23635ef-4bfa-4aae-b825-e0d7ad145a4d", "content-length": "729", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164712025-11-13T08:08:04.183ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
164722025-11-13T08:08:04.184ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "251f664c-1b79-44c4-9ef6-8e692d575f0c", "content-length": "728", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164732025-11-13T08:08:04.184ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
164742025-11-13T08:08:04.184ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
164752025-11-13T08:08:04.186ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164762025-11-13T08:08:04.188ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
164772025-11-13T08:08:04.188ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
164782025-11-13T08:08:04.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
164792025-11-13T08:08:04.190ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
164802025-11-13T08:08:04.192ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
164812025-11-13T08:08:04.193ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
164822025-11-13T08:08:04.193ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
164832025-11-13T08:08:04.225ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
164842025-11-13T08:08:04.225ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
164852025-11-13T08:08:04.284ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a8dd9c8-8295-4937-b96e-1be63b3e0011", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164862025-11-13T08:08:04.286ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
164872025-11-13T08:08:04.286ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
164882025-11-13T08:08:04.286ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
164892025-11-13T08:08:04.286ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
164902025-11-13T08:08:04.286ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
164912025-11-13T08:08:04.286ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
164922025-11-13T08:08:04.286ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
164932025-11-13T08:08:04.286ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
164942025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "07104bf2-c2cf-4947-8e42-e3d42aaffa20", "content-length": "64", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164952025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "4afa911c-20bf-4a3a-8e36-dd6e1ea90905", "content-length": "64", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164962025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
164972025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "e059fc79-ae1a-40c1-b444-e445515f0e18", "content-length": "64", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164982025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e57b99f-b5b7-42c9-9c70-4522e208636f", "content-length": "64", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
164992025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
165002025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
165012025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
165022025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bfb7ab1-577e-433d-aab0-3825ea620f9b", "content-length": "64", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165032025-11-13T08:08:04.287ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
165042025-11-13T08:08:04.325ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4dda0b1-ec58-4950-b763-27f4ef304d48", "content-length": "402303", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165052025-11-13T08:08:04.326ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "d97c5e98-edf8-414f-8fc4-ee00d1fa4c2a", "content-length": "401905", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165062025-11-13T08:08:04.326ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1aad5bc-0bb7-478a-9b11-3d3b6b392433", "content-length": "425486", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165072025-11-13T08:08:04.327ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "11bda668-3fd0-498c-b1c2-473d975aeca7", "content-length": "409610", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165082025-11-13T08:08:04.330ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "335f9d2e-9877-4747-84bb-dee200770274", "content-length": "402962", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165092025-11-13T08:08:04.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
165102025-11-13T08:08:04.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
165112025-11-13T08:08:04.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
165122025-11-13T08:08:04.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
165132025-11-13T08:08:04.332ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
165142025-11-13T08:08:04.368ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
165152025-11-13T08:08:04.368ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
165162025-11-13T08:08:04.372ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb1ca850-309e-4499-a8cf-ff7ce17ed5ed", "content-length": "124", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165172025-11-13T08:08:04.374ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
165182025-11-13T08:08:04.465ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
165192025-11-13T08:08:04.465ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
165202025-11-13T08:08:04.466ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4674d46-678e-4490-8d1f-c548644fa58e", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165212025-11-13T08:08:04.466ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
165222025-11-13T08:08:04.467ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
165232025-11-13T08:08:04.467ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
165242025-11-13T08:08:04.468ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a81b26a-eb60-4ee8-844f-cffda96de9e5", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165252025-11-13T08:08:04.468ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
165262025-11-13T08:08:04.470ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
165272025-11-13T08:08:04.470ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
165282025-11-13T08:08:04.470ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4f726aa-9895-4998-93ab-42222c752588", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:08:04 GMT"} })
165292025-11-13T08:08:04.470ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
165302025-11-13T08:08:04.471ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
165312025-11-13T08:08:04.471ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
165322025-11-13T08:08:04.473ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165332025-11-13T08:08:04.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
165342025-11-13T08:08:04.475ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165352025-11-13T08:08:04.477ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
165362025-11-13T08:08:04.477ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165372025-11-13T08:08:04.477ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
165382025-11-13T08:08:04.693ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = d00f4add-2b1d-4d87-8ba8-788f041ad77f file = nexus/db-queries/src/db/datastore/inventory.rs:1613
165392025-11-13T08:08:04.693ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = d00f4add-2b1d-4d87-8ba8-788f041ad77f time_started = 2025-11-13 08:08:04.175482 UTC
165402025-11-13T08:08:04.693ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 48 reason = Dependency
165412025-11-13T08:08:04.693ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
165422025-11-13T08:08:04.693ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.034770013s iteration = 11 status = {"collection_id":"d00f4add-2b1d-4d87-8ba8-788f041ad77f","time_done":"2025-11-13 08:08:04.471481 UTC","time_started":"2025-11-13 08:08:04.175482 UTC"}
165432025-11-13T08:08:04.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
165442025-11-13T08:08:04.695ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165452025-11-13T08:08:04.699ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
165462025-11-13T08:08:04.699ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
165472025-11-13T08:08:04.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
165482025-11-13T08:08:04.788ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
165492025-11-13T08:08:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
165502025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 150.628092ms iteration = 48 status = {"Loaded":{"collection_id":"1fd1eb26-3b73-4aab-81aa-76414004f934","time_loaded":"2025-11-13T08:08:04.693558754Z","time_started":"2025-11-13T08:08:04.180006Z"}}
165512025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 14 reason = Dependency
165522025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 31 reason = Dependency
165532025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 24 reason = Dependency
165542025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
165552025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165562025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
165572025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
165582025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
165592025-11-13T08:08:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165602025-11-13T08:08:04.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165612025-11-13T08:08:04.846ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165622025-11-13T08:08:04.846ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165632025-11-13T08:08:04.849ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
165642025-11-13T08:08:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165652025-11-13T08:08:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
165662025-11-13T08:08:04.849ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
165672025-11-13T08:08:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165682025-11-13T08:08:04.849ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
165692025-11-13T08:08:04.850ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
165702025-11-13T08:08:04.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
165712025-11-13T08:08:04.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
165722025-11-13T08:08:04.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165732025-11-13T08:08:04.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165742025-11-13T08:08:04.851ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
165752025-11-13T08:08:04.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
165762025-11-13T08:08:04.853ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165772025-11-13T08:08:04.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
165782025-11-13T08:08:04.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
165792025-11-13T08:08:04.854ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
165802025-11-13T08:08:04.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
165812025-11-13T08:08:04.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
165822025-11-13T08:08:04.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
165832025-11-13T08:08:04.855ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
165842025-11-13T08:08:04.855ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
165852025-11-13T08:08:04.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
165862025-11-13T08:08:04.856ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
165872025-11-13T08:08:04.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
165882025-11-13T08:08:04.858ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165892025-11-13T08:08:04.860ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
165902025-11-13T08:08:04.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
165912025-11-13T08:08:04.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
165922025-11-13T08:08:04.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165932025-11-13T08:08:04.864ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
165942025-11-13T08:08:04.864ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
165952025-11-13T08:08:04.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
165962025-11-13T08:08:04.869ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
165972025-11-13T08:08:04.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
165982025-11-13T08:08:04.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
165992025-11-13T08:08:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166002025-11-13T08:08:04.873ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
166012025-11-13T08:08:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
166022025-11-13T08:08:04.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
166032025-11-13T08:08:04.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166042025-11-13T08:08:04.877ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
166052025-11-13T08:08:04.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
166062025-11-13T08:08:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
166072025-11-13T08:08:04.879ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
166082025-11-13T08:08:04.879ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 35.557839ms iteration = 14 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"1fd1eb26-3b73-4aab-81aa-76414004f934","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
166092025-11-13T08:08:04.908ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 64.639341ms iteration = 31 status = {"physical_disks_added":0}
166102025-11-13T08:08:04.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166112025-11-13T08:08:04.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166122025-11-13T08:08:04.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166132025-11-13T08:08:04.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
166142025-11-13T08:08:04.935ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166152025-11-13T08:08:04.935ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166162025-11-13T08:08:04.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166172025-11-13T08:08:04.938ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
166182025-11-13T08:08:04.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166192025-11-13T08:08:04.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166202025-11-13T08:08:04.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166212025-11-13T08:08:04.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166222025-11-13T08:08:04.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166232025-11-13T08:08:04.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166242025-11-13T08:08:04.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166252025-11-13T08:08:04.948ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
166262025-11-13T08:08:04.948ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166272025-11-13T08:08:04.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166282025-11-13T08:08:04.950ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166292025-11-13T08:08:04.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166302025-11-13T08:08:04.952ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166312025-11-13T08:08:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
166322025-11-13T08:08:04.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166332025-11-13T08:08:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166342025-11-13T08:08:04.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166352025-11-13T08:08:04.959ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
166362025-11-13T08:08:04.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166372025-11-13T08:08:04.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166382025-11-13T08:08:04.961ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166392025-11-13T08:08:04.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166402025-11-13T08:08:04.962ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166412025-11-13T08:08:04.965ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
166422025-11-13T08:08:04.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166432025-11-13T08:08:04.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166442025-11-13T08:08:04.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166452025-11-13T08:08:04.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166462025-11-13T08:08:04.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166472025-11-13T08:08:04.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
166482025-11-13T08:08:04.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166492025-11-13T08:08:04.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166502025-11-13T08:08:04.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166512025-11-13T08:08:04.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166522025-11-13T08:08:04.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166532025-11-13T08:08:04.978ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
166542025-11-13T08:08:04.978ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166552025-11-13T08:08:04.978ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166562025-11-13T08:08:04.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166572025-11-13T08:08:04.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166582025-11-13T08:08:05.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166592025-11-13T08:08:05.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166602025-11-13T08:08:05.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166612025-11-13T08:08:05.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
166622025-11-13T08:08:05.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166632025-11-13T08:08:05.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166642025-11-13T08:08:05.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166652025-11-13T08:08:05.011ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
166662025-11-13T08:08:05.011ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166672025-11-13T08:08:05.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166682025-11-13T08:08:05.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166692025-11-13T08:08:05.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166702025-11-13T08:08:05.017ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166712025-11-13T08:08:05.017ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166722025-11-13T08:08:05.018ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166732025-11-13T08:08:05.021ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
166742025-11-13T08:08:05.021ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166752025-11-13T08:08:05.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166762025-11-13T08:08:05.023ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166772025-11-13T08:08:05.026ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
166782025-11-13T08:08:05.026ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166792025-11-13T08:08:05.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166802025-11-13T08:08:05.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166812025-11-13T08:08:05.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166822025-11-13T08:08:05.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166832025-11-13T08:08:05.030ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166842025-11-13T08:08:05.031ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166852025-11-13T08:08:05.034ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
166862025-11-13T08:08:05.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166872025-11-13T08:08:05.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166882025-11-13T08:08:05.036ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166892025-11-13T08:08:05.036ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166902025-11-13T08:08:05.038ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166912025-11-13T08:08:05.041ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
166922025-11-13T08:08:05.042ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166932025-11-13T08:08:05.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166942025-11-13T08:08:05.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166952025-11-13T08:08:05.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
166962025-11-13T08:08:05.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166972025-11-13T08:08:05.049ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
166982025-11-13T08:08:05.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
166992025-11-13T08:08:05.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167002025-11-13T08:08:05.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167012025-11-13T08:08:05.051ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167022025-11-13T08:08:05.053ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167032025-11-13T08:08:05.055ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
167042025-11-13T08:08:05.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167052025-11-13T08:08:05.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167062025-11-13T08:08:05.057ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167072025-11-13T08:08:05.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167082025-11-13T08:08:05.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167092025-11-13T08:08:05.062ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
167102025-11-13T08:08:05.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167112025-11-13T08:08:05.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167122025-11-13T08:08:05.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167132025-11-13T08:08:05.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167142025-11-13T08:08:05.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167152025-11-13T08:08:05.067ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
167162025-11-13T08:08:05.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167172025-11-13T08:08:05.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167182025-11-13T08:08:05.101ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
167192025-11-13T08:08:05.101ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
167202025-11-13T08:08:05.101ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 68e109cb-2d08-43f1-b67d-d049bfa61029 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
167212025-11-13T08:08:05.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167222025-11-13T08:08:05.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167232025-11-13T08:08:05.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167242025-11-13T08:08:05.105ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
167252025-11-13T08:08:05.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
167262025-11-13T08:08:05.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
167272025-11-13T08:08:05.108ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
167282025-11-13T08:08:05.108ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
167292025-11-13T08:08:05.108ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 263.941549ms iteration = 24 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
167302025-11-13T08:08:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 430 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = cfcd6390-b76c-48ec-b86f-330e9f38ffe0 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
167312025-11-13T08:08:07.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 110 reason = Timeout
167322025-11-13T08:08:07.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167332025-11-13T08:08:07.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167342025-11-13T08:08:07.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167352025-11-13T08:08:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
167362025-11-13T08:08:07.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167372025-11-13T08:08:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167382025-11-13T08:08:07.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
167392025-11-13T08:08:07.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.676261ms iteration = 110 status = {"config_updated":false}
167402025-11-13T08:08:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 56 reason = Timeout
167412025-11-13T08:08:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 111 reason = Timeout
167422025-11-13T08:08:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167432025-11-13T08:08:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167442025-11-13T08:08:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167452025-11-13T08:08:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167462025-11-13T08:08:12.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167472025-11-13T08:08:12.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167482025-11-13T08:08:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
167492025-11-13T08:08:12.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167502025-11-13T08:08:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167512025-11-13T08:08:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
167522025-11-13T08:08:12.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167532025-11-13T08:08:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167542025-11-13T08:08:12.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167552025-11-13T08:08:12.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167562025-11-13T08:08:12.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
167572025-11-13T08:08:12.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.199531ms iteration = 111 status = {"config_updated":false}
167582025-11-13T08:08:12.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167592025-11-13T08:08:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
167602025-11-13T08:08:12.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
167612025-11-13T08:08:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
167622025-11-13T08:08:12.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 56 reason = Timeout
167632025-11-13T08:08:12.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
167642025-11-13T08:08:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
167652025-11-13T08:08:12.654ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
167662025-11-13T08:08:12.654ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.789527ms iteration = 56 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
167672025-11-13T08:08:12.680ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
167682025-11-13T08:08:12.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
167692025-11-13T08:08:12.682ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 40.468406ms iteration = 56 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
167702025-11-13T08:08:15.473ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:37654
167712025-11-13T08:08:15.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9b6c37e-f6ef-402e-a756-62197a4ee436 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
167722025-11-13T08:08:15.474ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9b6c37e-f6ef-402e-a756-62197a4ee436 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
167732025-11-13T08:08:15.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9b6c37e-f6ef-402e-a756-62197a4ee436 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
167742025-11-13T08:08:15.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3128 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9b6c37e-f6ef-402e-a756-62197a4ee436 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
167752025-11-13T08:08:15.477ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 9bb39582-e11c-47a9-8e70-08243fad8678 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
167762025-11-13T08:08:15.477ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 9bb39582-e11c-47a9-8e70-08243fad8678 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
167772025-11-13T08:08:15.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 9bb39582-e11c-47a9-8e70-08243fad8678 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
167782025-11-13T08:08:15.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2079 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 9bb39582-e11c-47a9-8e70-08243fad8678 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
167792025-11-13T08:08:15.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 273 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 23f9e8ac-a4b6-4b83-a261-56f4f873a65a response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
167802025-11-13T08:08:17.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 49 reason = Timeout
167812025-11-13T08:08:17.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 112 reason = Timeout
167822025-11-13T08:08:17.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
167832025-11-13T08:08:17.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167842025-11-13T08:08:17.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
167852025-11-13T08:08:17.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167862025-11-13T08:08:17.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167872025-11-13T08:08:17.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167882025-11-13T08:08:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
167892025-11-13T08:08:17.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
167902025-11-13T08:08:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
167912025-11-13T08:08:17.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
167922025-11-13T08:08:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
167932025-11-13T08:08:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
167942025-11-13T08:08:17.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
167952025-11-13T08:08:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.759723ms iteration = 112 status = {"config_updated":false}
167962025-11-13T08:08:17.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1fd1eb26-3b73-4aab-81aa-76414004f934 original_time_started = 2025-11-13 08:08:04.180006 UTC
167972025-11-13T08:08:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.884655ms iteration = 49 status = {"Loaded":{"collection_id":"1fd1eb26-3b73-4aab-81aa-76414004f934","time_loaded":"2025-11-13T08:08:17.626550055Z","time_started":"2025-11-13T08:08:04.180006Z"}}
167982025-11-13T08:08:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 38 reason = Timeout
167992025-11-13T08:08:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168002025-11-13T08:08:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 38 reason = Timeout
168012025-11-13T08:08:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
168022025-11-13T08:08:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168032025-11-13T08:08:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
168042025-11-13T08:08:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168052025-11-13T08:08:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168062025-11-13T08:08:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168072025-11-13T08:08:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168082025-11-13T08:08:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
168092025-11-13T08:08:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
168102025-11-13T08:08:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
168112025-11-13T08:08:17.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
168122025-11-13T08:08:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168132025-11-13T08:08:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.035099ms iteration = 38 status = "NoSitrep"
168142025-11-13T08:08:17.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168152025-11-13T08:08:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
168162025-11-13T08:08:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
168172025-11-13T08:08:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168182025-11-13T08:08:17.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
168192025-11-13T08:08:17.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
168202025-11-13T08:08:17.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.908135ms iteration = 38 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
168212025-11-13T08:08:22.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 57 reason = Timeout
168222025-11-13T08:08:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 113 reason = Timeout
168232025-11-13T08:08:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168242025-11-13T08:08:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
168252025-11-13T08:08:22.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168262025-11-13T08:08:22.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
168272025-11-13T08:08:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168282025-11-13T08:08:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168292025-11-13T08:08:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
168302025-11-13T08:08:22.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
168312025-11-13T08:08:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
168322025-11-13T08:08:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
168332025-11-13T08:08:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168342025-11-13T08:08:22.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168352025-11-13T08:08:22.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
168362025-11-13T08:08:22.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.503097ms iteration = 113 status = {"config_updated":false}
168372025-11-13T08:08:22.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168382025-11-13T08:08:22.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168392025-11-13T08:08:22.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168402025-11-13T08:08:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
168412025-11-13T08:08:22.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168422025-11-13T08:08:22.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168432025-11-13T08:08:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 57 reason = Timeout
168442025-11-13T08:08:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
168452025-11-13T08:08:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
168462025-11-13T08:08:22.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
168472025-11-13T08:08:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
168482025-11-13T08:08:22.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 3.090201ms iteration = 57 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
168492025-11-13T08:08:22.651ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
168502025-11-13T08:08:22.651ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.400589ms iteration = 57 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
168512025-11-13T08:08:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 311 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 3ec589a1-15eb-4359-9132-53396466fa59 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
168522025-11-13T08:08:27.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 114 reason = Timeout
168532025-11-13T08:08:27.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
168542025-11-13T08:08:27.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
168552025-11-13T08:08:27.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168562025-11-13T08:08:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
168572025-11-13T08:08:27.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
168582025-11-13T08:08:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
168592025-11-13T08:08:27.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
168602025-11-13T08:08:27.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.911026ms iteration = 114 status = {"config_updated":false}
168612025-11-13T08:08:30.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 53cb998a-3207-445a-b39e-7ff5808e33fe uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
168622025-11-13T08:08:30.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 53cb998a-3207-445a-b39e-7ff5808e33fe roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
168632025-11-13T08:08:30.474ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 53cb998a-3207-445a-b39e-7ff5808e33fe resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
168642025-11-13T08:08:30.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3024 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 53cb998a-3207-445a-b39e-7ff5808e33fe response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
168652025-11-13T08:08:30.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 35335969-6da0-48cf-88b2-f5b46b275ad9 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
168662025-11-13T08:08:30.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 35335969-6da0-48cf-88b2-f5b46b275ad9 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
168672025-11-13T08:08:30.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 35335969-6da0-48cf-88b2-f5b46b275ad9 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
168682025-11-13T08:08:30.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2554 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 35335969-6da0-48cf-88b2-f5b46b275ad9 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
168692025-11-13T08:08:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 20 reason = Timeout
168702025-11-13T08:08:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 20 reason = Timeout
168712025-11-13T08:08:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
168722025-11-13T08:08:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 20 reason = Timeout
168732025-11-13T08:08:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
168742025-11-13T08:08:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
168752025-11-13T08:08:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
168762025-11-13T08:08:32.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6520, tv_nsec: 963732561 } })
168772025-11-13T08:08:32.612ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6520, tv_nsec: 964440535 } })
168782025-11-13T08:08:32.612ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
168792025-11-13T08:08:32.620ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 12.521866ms iteration = 20 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
168802025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 30 reason = Timeout
168812025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 50 reason = Timeout
168822025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
168832025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 32 reason = Timeout
168842025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 58 reason = Timeout
168852025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
168862025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 115 reason = Timeout
168872025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
168882025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
168892025-11-13T08:08:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
168902025-11-13T08:08:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
168912025-11-13T08:08:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
168922025-11-13T08:08:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
168932025-11-13T08:08:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
168942025-11-13T08:08:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
168952025-11-13T08:08:32.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168962025-11-13T08:08:32.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168972025-11-13T08:08:32.632ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168982025-11-13T08:08:32.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168992025-11-13T08:08:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
169002025-11-13T08:08:32.636ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
169012025-11-13T08:08:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
169022025-11-13T08:08:32.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
169032025-11-13T08:08:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
169042025-11-13T08:08:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
169052025-11-13T08:08:32.637ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
169062025-11-13T08:08:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
169072025-11-13T08:08:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
169082025-11-13T08:08:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
169092025-11-13T08:08:32.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
169102025-11-13T08:08:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
169112025-11-13T08:08:32.640ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
169122025-11-13T08:08:32.640ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 14.310732ms iteration = 115 status = {"config_updated":false}
169132025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
169142025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 20 reason = Timeout
169152025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
169162025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 20 reason = Timeout
169172025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
169182025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
169192025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 20 reason = Timeout
169202025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 20 reason = Timeout
169212025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
169222025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 20 reason = Timeout
169232025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
169242025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
169252025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
169262025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 20 reason = Timeout
169272025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
169282025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 58 reason = Timeout
169292025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
169302025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 30 reason = Timeout
169312025-11-13T08:08:32.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
169322025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
169332025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
169342025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
169352025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
169362025-11-13T08:08:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
169372025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
169382025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
169392025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
169402025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
169412025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
169422025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
169432025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169442025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1fd1eb26-3b73-4aab-81aa-76414004f934 original_time_started = 2025-11-13 08:08:04.180006 UTC
169452025-11-13T08:08:32.643ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 17.474294ms iteration = 50 status = {"Loaded":{"collection_id":"1fd1eb26-3b73-4aab-81aa-76414004f934","time_loaded":"2025-11-13T08:08:32.626428681Z","time_started":"2025-11-13T08:08:04.180006Z"}}
169462025-11-13T08:08:32.645ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169472025-11-13T08:08:32.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
169482025-11-13T08:08:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
169492025-11-13T08:08:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
169502025-11-13T08:08:32.647ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
169512025-11-13T08:08:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
169522025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 20 reason = Timeout
169532025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
169542025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 20 reason = Timeout
169552025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 20 reason = Timeout
169562025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 20 reason = Timeout
169572025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
169582025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 20 reason = Timeout
169592025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
169602025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 39 reason = Timeout
169612025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
169622025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 39 reason = Timeout
169632025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
169642025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
169652025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 20 reason = Timeout
169662025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
169672025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
169682025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
169692025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
169702025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
169712025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
169722025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
169732025-11-13T08:08:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
169742025-11-13T08:08:32.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6521, tv_nsec: 1243259 } })
169752025-11-13T08:08:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
169762025-11-13T08:08:32.649ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
169772025-11-13T08:08:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
169782025-11-13T08:08:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
169792025-11-13T08:08:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
169802025-11-13T08:08:32.650ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
169812025-11-13T08:08:32.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169822025-11-13T08:08:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
169832025-11-13T08:08:32.652ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169842025-11-13T08:08:32.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
169852025-11-13T08:08:32.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
169862025-11-13T08:08:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
169872025-11-13T08:08:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
169882025-11-13T08:08:32.656ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
169892025-11-13T08:08:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
169902025-11-13T08:08:32.656ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 8.66245ms iteration = 39 status = "NoSitrep"
169912025-11-13T08:08:32.658ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 10.500927ms iteration = 20 status = {"errors":[],"garbage_collect_requested":[]}
169922025-11-13T08:08:32.664ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
169932025-11-13T08:08:32.664ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 37.823683ms iteration = 30 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
169942025-11-13T08:08:32.733ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169952025-11-13T08:08:32.735ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
169962025-11-13T08:08:32.735ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
169972025-11-13T08:08:32.735ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
169982025-11-13T08:08:32.736ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 87.99066ms iteration = 20 status = {"errors":[],"finish_invoked_ok":[]}
169992025-11-13T08:08:32.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
170002025-11-13T08:08:32.736ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170012025-11-13T08:08:32.739ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
170022025-11-13T08:08:32.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
170032025-11-13T08:08:32.739ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
170042025-11-13T08:08:32.739ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
170052025-11-13T08:08:32.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
170062025-11-13T08:08:32.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
170072025-11-13T08:08:32.741ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
170082025-11-13T08:08:32.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
170092025-11-13T08:08:32.741ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
170102025-11-13T08:08:32.741ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
170112025-11-13T08:08:32.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
170122025-11-13T08:08:32.741ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170132025-11-13T08:08:32.742ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
170142025-11-13T08:08:32.744ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 96.023038ms iteration = 20 status = {"errors":[],"requests_created_ok":[]}
170152025-11-13T08:08:32.744ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 102.380403ms iteration = 20 status = 0
170162025-11-13T08:08:32.745ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
170172025-11-13T08:08:32.745ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
170182025-11-13T08:08:32.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
170192025-11-13T08:08:32.747ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170202025-11-13T08:08:32.748ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170212025-11-13T08:08:32.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
170222025-11-13T08:08:32.753ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
170232025-11-13T08:08:32.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
170242025-11-13T08:08:32.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
170252025-11-13T08:08:32.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
170262025-11-13T08:08:32.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
170272025-11-13T08:08:32.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
170282025-11-13T08:08:32.754ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170292025-11-13T08:08:32.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
170302025-11-13T08:08:32.755ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
170312025-11-13T08:08:32.757ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
170322025-11-13T08:08:32.759ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 151.330686ms iteration = 20 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
170332025-11-13T08:08:32.759ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6521, tv_nsec: 111495186 } })
170342025-11-13T08:08:32.759ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
170352025-11-13T08:08:32.760ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170362025-11-13T08:08:32.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170372025-11-13T08:08:32.765ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
170382025-11-13T08:08:32.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
170392025-11-13T08:08:32.766ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
170402025-11-13T08:08:32.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
170412025-11-13T08:08:32.767ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170422025-11-13T08:08:32.769ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
170432025-11-13T08:08:32.770ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
170442025-11-13T08:08:32.770ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 122.228083ms iteration = 39 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
170452025-11-13T08:08:32.774ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
170462025-11-13T08:08:32.775ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "09fafc76-4eea-48e6-a116-77703981edba", "content-length": "78", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
170472025-11-13T08:08:32.775ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
170482025-11-13T08:08:32.775ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
170492025-11-13T08:08:32.775ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
170502025-11-13T08:08:32.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
170512025-11-13T08:08:32.775ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
170522025-11-13T08:08:32.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
170532025-11-13T08:08:32.776ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
170542025-11-13T08:08:32.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
170552025-11-13T08:08:32.778ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
170562025-11-13T08:08:32.778ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
170572025-11-13T08:08:32.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
170582025-11-13T08:08:32.783ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
170592025-11-13T08:08:32.783ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
170602025-11-13T08:08:32.783ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
170612025-11-13T08:08:32.784ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
170622025-11-13T08:08:32.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
170632025-11-13T08:08:32.784ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 157.573098ms iteration = 58 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
170642025-11-13T08:08:32.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
170652025-11-13T08:08:32.785ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
170662025-11-13T08:08:32.785ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
170672025-11-13T08:08:32.786ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6659ddcd-209a-41c8-8c59-454743191f46", "content-length": "26", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
170682025-11-13T08:08:32.786ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
170692025-11-13T08:08:32.788ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
170702025-11-13T08:08:32.815ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
170712025-11-13T08:08:32.816ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
170722025-11-13T08:08:32.816ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 173.589843ms iteration = 20 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
170732025-11-13T08:08:32.823ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
170742025-11-13T08:08:32.823ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 180.856225ms iteration = 20 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
170752025-11-13T08:08:32.824ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170762025-11-13T08:08:32.825ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 182.865015ms iteration = 58 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
170772025-11-13T08:08:32.828ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
170782025-11-13T08:08:32.828ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
170792025-11-13T08:08:32.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
170802025-11-13T08:08:32.832ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 189.861742ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
170812025-11-13T08:08:32.839ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
170822025-11-13T08:08:32.839ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
170832025-11-13T08:08:32.840ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef9498d2-d26d-4aa6-9138-9b2869ef1b40", "content-length": "26", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
170842025-11-13T08:08:32.840ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
170852025-11-13T08:08:32.841ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
170862025-11-13T08:08:32.841ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
170872025-11-13T08:08:32.844ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
170882025-11-13T08:08:32.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
170892025-11-13T08:08:32.845ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
170902025-11-13T08:08:32.845ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
170912025-11-13T08:08:32.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
170922025-11-13T08:08:32.852ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
170932025-11-13T08:08:32.852ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
170942025-11-13T08:08:32.852ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8a5e3c0-f105-467d-93eb-87c59f163ca2", "content-length": "81", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
170952025-11-13T08:08:32.853ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
170962025-11-13T08:08:32.853ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
170972025-11-13T08:08:32.853ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 205.334827ms iteration = 20 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
170982025-11-13T08:08:32.853ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc7d2aee-df6e-463f-89a6-a61017d84065", "content-length": "81", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
170992025-11-13T08:08:32.857ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 250.001425ms iteration = 20 status = {}
171002025-11-13T08:08:32.858ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 232.090683ms iteration = 32 status = {"physical_disks_added":0}
171012025-11-13T08:08:32.878ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6521, tv_nsec: 230970325 } })
171022025-11-13T08:08:32.887ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 239.0455ms iteration = 20 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
171032025-11-13T08:08:32.888ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
171042025-11-13T08:08:32.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
171052025-11-13T08:08:32.892ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
171062025-11-13T08:08:32.893ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
171072025-11-13T08:08:32.893ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fa44e7b-13db-402e-b47c-c060866c955f", "content-length": "81", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
171082025-11-13T08:08:32.895ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 246.743841ms iteration = 20 status = {"disabled":false,"errors":[],"sps":[]}
171092025-11-13T08:08:32.898ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
171102025-11-13T08:08:32.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
171112025-11-13T08:08:32.900ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171122025-11-13T08:08:32.902ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
171132025-11-13T08:08:32.902ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
171142025-11-13T08:08:32.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
171152025-11-13T08:08:32.904ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171162025-11-13T08:08:32.906ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
171172025-11-13T08:08:32.906ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
171182025-11-13T08:08:32.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
171192025-11-13T08:08:32.956ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
171202025-11-13T08:08:32.956ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171212025-11-13T08:08:32.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171222025-11-13T08:08:32.957ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
171232025-11-13T08:08:32.957ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171242025-11-13T08:08:32.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171252025-11-13T08:08:32.961ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
171262025-11-13T08:08:32.961ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
171272025-11-13T08:08:32.961ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "afba9445-55d2-48d2-ab5d-2dabe123c757", "content-length": "691", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
171282025-11-13T08:08:32.961ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
171292025-11-13T08:08:32.961ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
171302025-11-13T08:08:32.961ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
171312025-11-13T08:08:32.961ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "57477bc3-d11d-49a5-8944-3fe93bafd25f", "content-length": "372", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
171322025-11-13T08:08:32.961ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
171332025-11-13T08:08:32.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
171342025-11-13T08:08:32.963ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 321.4192ms iteration = 20 status = {}
171352025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
171362025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
171372025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
171382025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
171392025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
171402025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
171412025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
171422025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
171432025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
171442025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
171452025-11-13T08:08:32.970ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
171462025-11-13T08:08:32.970ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171472025-11-13T08:08:32.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171482025-11-13T08:08:32.973ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
171492025-11-13T08:08:32.973ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
171502025-11-13T08:08:32.975ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
171512025-11-13T08:08:32.975ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
171522025-11-13T08:08:32.977ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "ec2bb614-0236-4c3a-a31d-c960b72d6b2b", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
171532025-11-13T08:08:32.977ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
171542025-11-13T08:08:32.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
171552025-11-13T08:08:32.980ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
171562025-11-13T08:08:32.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
171572025-11-13T08:08:32.982ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
171582025-11-13T08:08:32.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
171592025-11-13T08:08:32.986ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
171602025-11-13T08:08:32.986ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
171612025-11-13T08:08:32.986ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "29b1532b-5068-4c59-a7ec-9208102abea8", "date": "Thu, 13 Nov 2025 08:08:32 GMT"} })
171622025-11-13T08:08:32.988ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 346.315308ms iteration = 30 status = {}
171632025-11-13T08:08:33.135ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "6f28c021-e324-4445-b4d1-6dfd674a1d30", "date": "Thu, 13 Nov 2025 08:08:33 GMT"} })
171642025-11-13T08:08:33.135ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
171652025-11-13T08:08:33.135ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
171662025-11-13T08:08:33.140ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e8fb2b5-b406-43fa-83ba-579b6086c198", "content-length": "682", "date": "Thu, 13 Nov 2025 08:08:33 GMT"} })
171672025-11-13T08:08:33.140ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171682025-11-13T08:08:33.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171692025-11-13T08:08:33.145ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171702025-11-13T08:08:33.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171712025-11-13T08:08:33.148ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171722025-11-13T08:08:33.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171732025-11-13T08:08:33.150ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
171742025-11-13T08:08:33.150ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
171752025-11-13T08:08:33.157ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 515.069656ms iteration = 20 status = {}
171762025-11-13T08:08:35.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 314 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 6b4e589e-0a74-40b7-8b71-fb3dd980a079 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
171772025-11-13T08:08:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 116 reason = Timeout
171782025-11-13T08:08:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
171792025-11-13T08:08:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
171802025-11-13T08:08:37.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171812025-11-13T08:08:37.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
171822025-11-13T08:08:37.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
171832025-11-13T08:08:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
171842025-11-13T08:08:37.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
171852025-11-13T08:08:37.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.200962ms iteration = 116 status = {"config_updated":false}
171862025-11-13T08:08:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 59 reason = Timeout
171872025-11-13T08:08:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 117 reason = Timeout
171882025-11-13T08:08:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
171892025-11-13T08:08:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
171902025-11-13T08:08:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
171912025-11-13T08:08:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
171922025-11-13T08:08:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171932025-11-13T08:08:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171942025-11-13T08:08:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
171952025-11-13T08:08:42.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
171962025-11-13T08:08:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
171972025-11-13T08:08:42.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
171982025-11-13T08:08:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
171992025-11-13T08:08:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172002025-11-13T08:08:42.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
172012025-11-13T08:08:42.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.874566ms iteration = 117 status = {"config_updated":false}
172022025-11-13T08:08:42.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172032025-11-13T08:08:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172042025-11-13T08:08:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172052025-11-13T08:08:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
172062025-11-13T08:08:42.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172072025-11-13T08:08:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172082025-11-13T08:08:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 59 reason = Timeout
172092025-11-13T08:08:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
172102025-11-13T08:08:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
172112025-11-13T08:08:42.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
172122025-11-13T08:08:42.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
172132025-11-13T08:08:42.657ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
172142025-11-13T08:08:42.657ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 30.783525ms iteration = 59 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
172152025-11-13T08:08:42.680ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.506357ms iteration = 59 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
172162025-11-13T08:08:45.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 87e9b701-3488-4636-ad53-16866c7a9834 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
172172025-11-13T08:08:45.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 87e9b701-3488-4636-ad53-16866c7a9834 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
172182025-11-13T08:08:45.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 87e9b701-3488-4636-ad53-16866c7a9834 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
172192025-11-13T08:08:45.477ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 4559 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 87e9b701-3488-4636-ad53-16866c7a9834 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
172202025-11-13T08:08:45.477ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = e19ac8c9-2c8c-4b1d-a92d-16eca0e2296e uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
172212025-11-13T08:08:45.478ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = e19ac8c9-2c8c-4b1d-a92d-16eca0e2296e roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
172222025-11-13T08:08:45.478ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = e19ac8c9-2c8c-4b1d-a92d-16eca0e2296e resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
172232025-11-13T08:08:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 243 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = e79d4433-591e-4fac-bb60-44a6dcd28fbe response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
172242025-11-13T08:08:45.481ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3512 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = e19ac8c9-2c8c-4b1d-a92d-16eca0e2296e response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
172252025-11-13T08:08:47.625ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 51 reason = Timeout
172262025-11-13T08:08:47.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 118 reason = Timeout
172272025-11-13T08:08:47.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
172282025-11-13T08:08:47.625ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
172292025-11-13T08:08:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
172302025-11-13T08:08:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
172312025-11-13T08:08:47.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172322025-11-13T08:08:47.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172332025-11-13T08:08:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
172342025-11-13T08:08:47.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
172352025-11-13T08:08:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
172362025-11-13T08:08:47.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
172372025-11-13T08:08:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
172382025-11-13T08:08:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
172392025-11-13T08:08:47.633ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1fd1eb26-3b73-4aab-81aa-76414004f934 original_time_started = 2025-11-13 08:08:04.180006 UTC
172402025-11-13T08:08:47.633ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.64227ms iteration = 51 status = {"Loaded":{"collection_id":"1fd1eb26-3b73-4aab-81aa-76414004f934","time_loaded":"2025-11-13T08:08:47.625854008Z","time_started":"2025-11-13T08:08:04.180006Z"}}
172412025-11-13T08:08:47.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
172422025-11-13T08:08:47.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.65173ms iteration = 118 status = {"config_updated":false}
172432025-11-13T08:08:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 40 reason = Timeout
172442025-11-13T08:08:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 40 reason = Timeout
172452025-11-13T08:08:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
172462025-11-13T08:08:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
172472025-11-13T08:08:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
172482025-11-13T08:08:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
172492025-11-13T08:08:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
172502025-11-13T08:08:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
172512025-11-13T08:08:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172522025-11-13T08:08:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172532025-11-13T08:08:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
172542025-11-13T08:08:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
172552025-11-13T08:08:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
172562025-11-13T08:08:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
172572025-11-13T08:08:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.456117ms iteration = 40 status = "NoSitrep"
172582025-11-13T08:08:47.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
172592025-11-13T08:08:47.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
172602025-11-13T08:08:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
172612025-11-13T08:08:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
172622025-11-13T08:08:47.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172632025-11-13T08:08:47.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
172642025-11-13T08:08:47.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
172652025-11-13T08:08:47.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.712551ms iteration = 40 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
172662025-11-13T08:08:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 60 reason = Timeout
172672025-11-13T08:08:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 119 reason = Timeout
172682025-11-13T08:08:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172692025-11-13T08:08:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
172702025-11-13T08:08:52.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172712025-11-13T08:08:52.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
172722025-11-13T08:08:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172732025-11-13T08:08:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172742025-11-13T08:08:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
172752025-11-13T08:08:52.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
172762025-11-13T08:08:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
172772025-11-13T08:08:52.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172782025-11-13T08:08:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
172792025-11-13T08:08:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172802025-11-13T08:08:52.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
172812025-11-13T08:08:52.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.844215ms iteration = 119 status = {"config_updated":false}
172822025-11-13T08:08:52.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172832025-11-13T08:08:52.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172842025-11-13T08:08:52.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172852025-11-13T08:08:52.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
172862025-11-13T08:08:52.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
172872025-11-13T08:08:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
172882025-11-13T08:08:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 60 reason = Timeout
172892025-11-13T08:08:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
172902025-11-13T08:08:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
172912025-11-13T08:08:52.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
172922025-11-13T08:08:52.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
172932025-11-13T08:08:52.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.821195ms iteration = 60 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
172942025-11-13T08:08:52.649ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
172952025-11-13T08:08:52.649ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.201957ms iteration = 60 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
172962025-11-13T08:08:55.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 365 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 78771118-1d90-42b1-9b7e-e06fdf4dd32b response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
172972025-11-13T08:08:57.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 120 reason = Timeout
172982025-11-13T08:08:57.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
172992025-11-13T08:08:57.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
173002025-11-13T08:08:57.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173012025-11-13T08:08:57.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
173022025-11-13T08:08:57.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
173032025-11-13T08:08:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
173042025-11-13T08:08:57.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
173052025-11-13T08:08:57.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.423297ms iteration = 120 status = {"config_updated":false}
173062025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 11 reason = Timeout
173072025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 11 reason = Timeout
173082025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 11 reason = Timeout
173092025-11-13T08:09:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 07:59:02.607088646 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
173102025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 11 reason = Timeout
173112025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
173122025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
173132025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
173142025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 11 reason = Timeout
173152025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
173162025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
173172025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
173182025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 11 reason = Timeout
173192025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 21 reason = Timeout
173202025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 21 reason = Timeout
173212025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 13 reason = Timeout
173222025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
173232025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 13 reason = Timeout
173242025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 21 reason = Timeout
173252025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
173262025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
173272025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
173282025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
173292025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 12 reason = Timeout
173302025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
173312025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
173322025-11-13T08:09:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
173332025-11-13T08:09:02.607ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = ExternalNetworking step id = Ensure
173342025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
173352025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
173362025-11-13T08:09:02.607ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6550, tv_nsec: 960259128 } })
173372025-11-13T08:09:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
173382025-11-13T08:09:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6550, tv_nsec: 960411021 } })
173392025-11-13T08:09:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
173402025-11-13T08:09:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
173412025-11-13T08:09:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.583111ms iteration = 11 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
173422025-11-13T08:09:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
173432025-11-13T08:09:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.568811ms iteration = 11 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
173442025-11-13T08:09:02.608ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
173452025-11-13T08:09:02.608ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
173462025-11-13T08:09:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
173472025-11-13T08:09:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
173482025-11-13T08:09:02.610ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 07:59:02.607088646 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
173492025-11-13T08:09:02.610ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 3.836055ms iteration = 11 status = {"expiration":"2025-11-13T07:59:02.607088646Z","pruned":[]}
173502025-11-13T08:09:02.610ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Timeout
173512025-11-13T08:09:02.610ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
173522025-11-13T08:09:02.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
173532025-11-13T08:09:02.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173542025-11-13T08:09:02.615ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173552025-11-13T08:09:02.615ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
173562025-11-13T08:09:02.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
173572025-11-13T08:09:02.616ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6550, tv_nsec: 969217624 } })
173582025-11-13T08:09:02.617ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6550, tv_nsec: 969323476 } })
173592025-11-13T08:09:02.617ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
173602025-11-13T08:09:02.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173612025-11-13T08:09:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
173622025-11-13T08:09:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 15 reason = Timeout
173632025-11-13T08:09:02.618ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 11.338843ms iteration = 21 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
173642025-11-13T08:09:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
173652025-11-13T08:09:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 11 reason = Timeout
173662025-11-13T08:09:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
173672025-11-13T08:09:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
173682025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
173692025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 25 reason = Timeout
173702025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
173712025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
173722025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
173732025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
173742025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
173752025-11-13T08:09:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
173762025-11-13T08:09:02.620ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
173772025-11-13T08:09:02.620ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
173782025-11-13T08:09:02.620ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
173792025-11-13T08:09:02.620ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
173802025-11-13T08:09:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
173812025-11-13T08:09:02.623ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173822025-11-13T08:09:02.623ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
173832025-11-13T08:09:02.623ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
173842025-11-13T08:09:02.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
173852025-11-13T08:09:02.624ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
173862025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
173872025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
173882025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
173892025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
173902025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
173912025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 31 reason = Timeout
173922025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
173932025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 52 reason = Timeout
173942025-11-13T08:09:02.625ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 33 reason = Timeout
173952025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
173962025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
173972025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 61 reason = Timeout
173982025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 121 reason = Timeout
173992025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
174002025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
174012025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
174022025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
174032025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
174042025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
174052025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
174062025-11-13T08:09:02.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
174072025-11-13T08:09:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
174082025-11-13T08:09:02.627ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174092025-11-13T08:09:02.627ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
174102025-11-13T08:09:02.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
174112025-11-13T08:09:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
174122025-11-13T08:09:02.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174132025-11-13T08:09:02.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174142025-11-13T08:09:02.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174152025-11-13T08:09:02.629ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174162025-11-13T08:09:02.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174172025-11-13T08:09:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
174182025-11-13T08:09:02.630ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
174192025-11-13T08:09:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
174202025-11-13T08:09:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
174212025-11-13T08:09:02.634ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
174222025-11-13T08:09:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
174232025-11-13T08:09:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
174242025-11-13T08:09:02.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
174252025-11-13T08:09:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
174262025-11-13T08:09:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
174272025-11-13T08:09:02.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
174282025-11-13T08:09:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
174292025-11-13T08:09:02.636ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
174302025-11-13T08:09:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
174312025-11-13T08:09:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
174322025-11-13T08:09:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
174332025-11-13T08:09:02.637ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
174342025-11-13T08:09:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
174352025-11-13T08:09:02.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
174362025-11-13T08:09:02.639ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
174372025-11-13T08:09:02.639ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
174382025-11-13T08:09:02.639ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
174392025-11-13T08:09:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
174402025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 3 reason = Timeout
174412025-11-13T08:09:02.643ZDEBGnexus (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
174422025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174432025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174442025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 21 reason = Timeout
174452025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
174462025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 21 reason = Timeout
174472025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
174482025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
174492025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 21 reason = Timeout
174502025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
174512025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
174522025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 11 reason = Timeout
174532025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
174542025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 21 reason = Timeout
174552025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
174562025-11-13T08:09:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
174572025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 11 reason = Timeout
174582025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
174592025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
174602025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 61 reason = Timeout
174612025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
174622025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 21 reason = Timeout
174632025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
174642025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
174652025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 21 reason = Timeout
174662025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
174672025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
174682025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 31 reason = Timeout
174692025-11-13T08:09:02.644ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
174702025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
174712025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
174722025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174732025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
174742025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
174752025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
174762025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
174772025-11-13T08:09:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
174782025-11-13T08:09:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
174792025-11-13T08:09:02.645ZDEBGnexus (BackgroundTasks): activating background_task = saga_recovery iteration = 2 reason = Timeout
174802025-11-13T08:09:02.645ZINFOnexus (SagaQuiesceHandle): saga recovery pass starting file = nexus/types/src/quiesce.rs:510
174812025-11-13T08:09:02.645ZDEBGnexus (SagaRecoverer): listing candidate sagas for recovery
174822025-11-13T08:09:02.645ZDEBGnexus (SagaRecoverer): roles roles = RoleSet { roles: {} }
174832025-11-13T08:09:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 11 reason = Timeout
174842025-11-13T08:09:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
174852025-11-13T08:09:02.646ZDEBGnexus (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a (built_in_user), .. }) resource = Database result = Ok(())
174862025-11-13T08:09:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
174872025-11-13T08:09:02.648ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
174882025-11-13T08:09:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
174892025-11-13T08:09:02.648ZDEBGnexus (BackgroundTasks): activating background_task = tuf_repo_pruner iteration = 3 reason = Timeout
174902025-11-13T08:09:02.648ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
174912025-11-13T08:09:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
174922025-11-13T08:09:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
174932025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
174942025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 3 reason = Timeout
174952025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
174962025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
174972025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
174982025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
174992025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 11 reason = Timeout
175002025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
175012025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 21 reason = Timeout
175022025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175032025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
175042025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 41 reason = Timeout
175052025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
175062025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 21 reason = Timeout
175072025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
175082025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 21 reason = Timeout
175092025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
175102025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 21 reason = Timeout
175112025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 41 reason = Timeout
175122025-11-13T08:09:02.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
175132025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 21 reason = Timeout
175142025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
175152025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 11 reason = Timeout
175162025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
175172025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 21 reason = Timeout
175182025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
175192025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 11 reason = Timeout
175202025-11-13T08:09:02.650ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
175212025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
175222025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
175232025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
175242025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
175252025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
175262025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
175272025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
175282025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
175292025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
175302025-11-13T08:09:02.650ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6551, tv_nsec: 2951958 } })
175312025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
175322025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
175332025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
175342025-11-13T08:09:02.650ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
175352025-11-13T08:09:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
175362025-11-13T08:09:02.653ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175372025-11-13T08:09:02.657ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 1fd1eb26-3b73-4aab-81aa-76414004f934 original_time_started = 2025-11-13 08:08:04.180006 UTC
175382025-11-13T08:09:02.657ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 31.914167ms iteration = 52 status = {"Loaded":{"collection_id":"1fd1eb26-3b73-4aab-81aa-76414004f934","time_loaded":"2025-11-13T08:09:02.625970557Z","time_started":"2025-11-13T08:08:04.180006Z"}}
175392025-11-13T08:09:02.657ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
175402025-11-13T08:09:02.657ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
175412025-11-13T08:09:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
175422025-11-13T08:09:02.658ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
175432025-11-13T08:09:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
175442025-11-13T08:09:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
175452025-11-13T08:09:02.661ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
175462025-11-13T08:09:02.663ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 17.758849ms iteration = 11 status = {"errors":[],"found_port_ok":[]}
175472025-11-13T08:09:02.665ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
175482025-11-13T08:09:02.665ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
175492025-11-13T08:09:02.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
175502025-11-13T08:09:02.666ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
175512025-11-13T08:09:02.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
175522025-11-13T08:09:02.682ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
175532025-11-13T08:09:02.682ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175542025-11-13T08:09:02.683ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
175552025-11-13T08:09:02.686ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
175562025-11-13T08:09:02.686ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
175572025-11-13T08:09:02.686ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
175582025-11-13T08:09:02.686ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
175592025-11-13T08:09:02.687ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
175602025-11-13T08:09:02.693ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
175612025-11-13T08:09:02.695ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175622025-11-13T08:09:02.695ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
175632025-11-13T08:09:02.698ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
175642025-11-13T08:09:02.698ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
175652025-11-13T08:09:02.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
175662025-11-13T08:09:02.700ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
175672025-11-13T08:09:02.702ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
175682025-11-13T08:09:02.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175692025-11-13T08:09:02.704ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
175702025-11-13T08:09:02.709ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
175712025-11-13T08:09:02.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
175722025-11-13T08:09:02.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
175732025-11-13T08:09:02.719ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
175742025-11-13T08:09:02.719ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = SledList step id = Fetch
175752025-11-13T08:09:02.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
175762025-11-13T08:09:02.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
175772025-11-13T08:09:02.722ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
175782025-11-13T08:09:02.722ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 72.352612ms iteration = 11 status = {"by_rx":{},"error":null}
175792025-11-13T08:09:02.728ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175802025-11-13T08:09:02.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
175812025-11-13T08:09:02.732ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
175822025-11-13T08:09:02.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
175832025-11-13T08:09:02.736ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175842025-11-13T08:09:02.742ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
175852025-11-13T08:09:02.742ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
175862025-11-13T08:09:02.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
175872025-11-13T08:09:02.747ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175882025-11-13T08:09:02.749ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
175892025-11-13T08:09:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
175902025-11-13T08:09:02.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175912025-11-13T08:09:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
175922025-11-13T08:09:02.750ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
175932025-11-13T08:09:02.750ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 101.055757ms iteration = 21 status = {"errors":[],"requests_created_ok":[]}
175942025-11-13T08:09:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
175952025-11-13T08:09:02.754ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
175962025-11-13T08:09:02.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
175972025-11-13T08:09:02.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
175982025-11-13T08:09:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
175992025-11-13T08:09:02.755ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
176002025-11-13T08:09:02.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
176012025-11-13T08:09:02.757ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176022025-11-13T08:09:02.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
176032025-11-13T08:09:02.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
176042025-11-13T08:09:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
176052025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
176062025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
176072025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
176082025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176092025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
176102025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
176112025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
176122025-11-13T08:09:02.762ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 155.609879ms iteration = 11 status = {"generation":2}
176132025-11-13T08:09:02.766ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
176142025-11-13T08:09:02.766ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
176152025-11-13T08:09:02.767ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
176162025-11-13T08:09:02.771ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
176172025-11-13T08:09:02.771ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
176182025-11-13T08:09:02.771ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 127.910635ms iteration = 11 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
176192025-11-13T08:09:02.772ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176202025-11-13T08:09:02.775ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176212025-11-13T08:09:02.775ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176222025-11-13T08:09:02.776ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176232025-11-13T08:09:02.776ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
176242025-11-13T08:09:02.776ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
176252025-11-13T08:09:02.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176262025-11-13T08:09:02.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
176272025-11-13T08:09:02.777ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
176282025-11-13T08:09:02.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
176292025-11-13T08:09:02.779ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176302025-11-13T08:09:02.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
176312025-11-13T08:09:02.780ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
176322025-11-13T08:09:02.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
176332025-11-13T08:09:02.780ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
176342025-11-13T08:09:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
176352025-11-13T08:09:02.781ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 131.523386ms iteration = 41 status = "NoSitrep"
176362025-11-13T08:09:02.781ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
176372025-11-13T08:09:02.781ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
176382025-11-13T08:09:02.781ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
176392025-11-13T08:09:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
176402025-11-13T08:09:02.783ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
176412025-11-13T08:09:02.783ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
176422025-11-13T08:09:02.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
176432025-11-13T08:09:02.783ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176442025-11-13T08:09:02.784ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
176452025-11-13T08:09:02.784ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
176462025-11-13T08:09:02.784ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
176472025-11-13T08:09:02.785ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
176482025-11-13T08:09:02.785ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 158.828182ms iteration = 121 status = {"config_updated":false}
176492025-11-13T08:09:02.785ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176502025-11-13T08:09:02.786ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176512025-11-13T08:09:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
176522025-11-13T08:09:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
176532025-11-13T08:09:02.791ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
176542025-11-13T08:09:02.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
176552025-11-13T08:09:02.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
176562025-11-13T08:09:02.791ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
176572025-11-13T08:09:02.791ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
176582025-11-13T08:09:02.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
176592025-11-13T08:09:02.792ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
176602025-11-13T08:09:02.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176612025-11-13T08:09:02.792ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
176622025-11-13T08:09:02.793ZINFOnexus (SagaRecoverer): listed in-progress sagas count = 0 file = nexus/src/app/background/tasks/saga_recovery.rs:473
176632025-11-13T08:09:02.793ZINFOnexus (SagaQuiesceHandle): saga recovery pass finished blueprint_id = Some(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint)) file = nexus/types/src/quiesce.rs:525 generation = 1
176642025-11-13T08:09:02.793ZDEBGnexus (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 147.436678ms iteration = 2 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
176652025-11-13T08:09:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
176662025-11-13T08:09:02.794ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176672025-11-13T08:09:02.798ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
176682025-11-13T08:09:02.798ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
176692025-11-13T08:09:02.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
176702025-11-13T08:09:02.802ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
176712025-11-13T08:09:02.802ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
176722025-11-13T08:09:02.802ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
176732025-11-13T08:09:02.802ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
176742025-11-13T08:09:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
176752025-11-13T08:09:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
176762025-11-13T08:09:02.806ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
176772025-11-13T08:09:02.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
176782025-11-13T08:09:02.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
176792025-11-13T08:09:02.811ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
176802025-11-13T08:09:02.811ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
176812025-11-13T08:09:02.811ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
176822025-11-13T08:09:02.811ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6551, tv_nsec: 163847211 } })
176832025-11-13T08:09:02.811ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
176842025-11-13T08:09:02.812ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176852025-11-13T08:09:02.812ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
176862025-11-13T08:09:02.813ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176872025-11-13T08:09:02.815ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 170.970941ms iteration = 21 status = 0
176882025-11-13T08:09:02.817ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
176892025-11-13T08:09:02.817ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
176902025-11-13T08:09:02.817ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 173.213606ms iteration = 61 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
176912025-11-13T08:09:02.817ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
176922025-11-13T08:09:02.817ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Fleet result = Ok(())
176932025-11-13T08:09:02.817ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
176942025-11-13T08:09:02.818ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
176952025-11-13T08:09:02.822ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
176962025-11-13T08:09:02.826ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
176972025-11-13T08:09:02.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
176982025-11-13T08:09:02.826ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
176992025-11-13T08:09:02.826ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
177002025-11-13T08:09:02.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
177012025-11-13T08:09:02.827ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
177022025-11-13T08:09:02.827ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
177032025-11-13T08:09:02.848ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
177042025-11-13T08:09:02.848ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 222.187959ms iteration = 61 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
177052025-11-13T08:09:02.858ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177062025-11-13T08:09:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177072025-11-13T08:09:02.860ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
177082025-11-13T08:09:02.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
177092025-11-13T08:09:02.864ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 214.734792ms iteration = 21 status = {"errors":[],"garbage_collect_requested":[]}
177102025-11-13T08:09:02.872ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 223.036785ms iteration = 21 status = {"errors":[],"finish_invoked_ok":[]}
177112025-11-13T08:09:02.873ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
177122025-11-13T08:09:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
177132025-11-13T08:09:02.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177142025-11-13T08:09:02.882ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
177152025-11-13T08:09:02.882ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 232.395548ms iteration = 11 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
177162025-11-13T08:09:02.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
177172025-11-13T08:09:02.882ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177182025-11-13T08:09:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
177192025-11-13T08:09:02.885ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
177202025-11-13T08:09:02.889ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
177212025-11-13T08:09:02.889ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 239.789324ms iteration = 41 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
177222025-11-13T08:09:02.890ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177232025-11-13T08:09:02.893ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
177242025-11-13T08:09:02.894ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
177252025-11-13T08:09:02.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
177262025-11-13T08:09:02.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
177272025-11-13T08:09:02.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
177282025-11-13T08:09:02.895ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177292025-11-13T08:09:02.896ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 253.101106ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
177302025-11-13T08:09:02.898ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
177312025-11-13T08:09:02.898ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177322025-11-13T08:09:02.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177332025-11-13T08:09:02.898ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177342025-11-13T08:09:02.901ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177352025-11-13T08:09:02.901ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
177362025-11-13T08:09:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177372025-11-13T08:09:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
177382025-11-13T08:09:02.902ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
177392025-11-13T08:09:02.902ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 258.946151ms iteration = 21 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
177402025-11-13T08:09:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = TargetReleaseConfig result = Ok(())
177412025-11-13T08:09:02.902ZDEBGnexus (BackgroundTasks): roles background_task = tuf_repo_pruner roles = RoleSet { roles: {} }
177422025-11-13T08:09:02.902ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_repo_pruner resource = Database result = Ok(())
177432025-11-13T08:09:02.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
177442025-11-13T08:09:02.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
177452025-11-13T08:09:02.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
177462025-11-13T08:09:02.904ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177472025-11-13T08:09:02.904ZINFOnexus (BackgroundTasks): tuf_repo_prune decision background_task = tuf_repo_pruner file = nexus/src/app/background/tasks/tuf_repo_pruner.rs:114 status = TufRepoPrunerStatus { nkeep_recent_releases: 3, nkeep_recent_uploads: 2, repos_keep_target_release: {}, repos_keep_recent_uploads: {}, repo_prune: None, other_repos_eligible_to_prune: {}, warnings: [] }
177482025-11-13T08:09:02.904ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_repo_pruner elapsed = 256.068374ms iteration = 3 status = {"nkeep_recent_releases":3,"nkeep_recent_uploads":2,"other_repos_eligible_to_prune":[],"repo_prune":null,"repos_keep_recent_uploads":[],"repos_keep_target_release":[],"warnings":[]}
177492025-11-13T08:09:02.904ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177502025-11-13T08:09:02.906ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177512025-11-13T08:09:02.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177522025-11-13T08:09:02.914ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
177532025-11-13T08:09:02.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
177542025-11-13T08:09:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
177552025-11-13T08:09:02.914ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177562025-11-13T08:09:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
177572025-11-13T08:09:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177582025-11-13T08:09:02.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177592025-11-13T08:09:02.926ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
177602025-11-13T08:09:02.926ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 307.822032ms iteration = 15 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"1fd1eb26-3b73-4aab-81aa-76414004f934","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
177612025-11-13T08:09:02.929ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
177622025-11-13T08:09:02.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177632025-11-13T08:09:02.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177642025-11-13T08:09:02.930ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177652025-11-13T08:09:02.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177662025-11-13T08:09:02.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177672025-11-13T08:09:02.934ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177682025-11-13T08:09:02.934ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177692025-11-13T08:09:02.936ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
177702025-11-13T08:09:02.936ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
177712025-11-13T08:09:02.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
177722025-11-13T08:09:02.937ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177732025-11-13T08:09:02.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177742025-11-13T08:09:02.939ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177752025-11-13T08:09:02.940ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177762025-11-13T08:09:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177772025-11-13T08:09:02.941ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
177782025-11-13T08:09:02.942ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 334.971035ms iteration = 11 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
177792025-11-13T08:09:02.943ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177802025-11-13T08:09:02.943ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177812025-11-13T08:09:02.945ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
177822025-11-13T08:09:02.945ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
177832025-11-13T08:09:02.945ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
177842025-11-13T08:09:02.945ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
177852025-11-13T08:09:02.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
177862025-11-13T08:09:02.948ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
177872025-11-13T08:09:02.948ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
177882025-11-13T08:09:02.948ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
177892025-11-13T08:09:02.948ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
177902025-11-13T08:09:02.949ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 341.839731ms iteration = 11 status = {"generation":1}
177912025-11-13T08:09:02.953ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
177922025-11-13T08:09:02.954ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
177932025-11-13T08:09:02.954ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 309.78543ms iteration = 21 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
177942025-11-13T08:09:02.954ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
177952025-11-13T08:09:02.954ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
177962025-11-13T08:09:02.954ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 336.18528ms iteration = 11 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
177972025-11-13T08:09:02.954ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
177982025-11-13T08:09:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
177992025-11-13T08:09:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
178002025-11-13T08:09:02.956ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178012025-11-13T08:09:02.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178022025-11-13T08:09:02.957ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178032025-11-13T08:09:02.958ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178042025-11-13T08:09:02.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178052025-11-13T08:09:02.961ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
178062025-11-13T08:09:02.961ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
178072025-11-13T08:09:02.961ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178082025-11-13T08:09:02.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
178092025-11-13T08:09:02.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178102025-11-13T08:09:02.965ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178112025-11-13T08:09:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178122025-11-13T08:09:02.971ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 321.95323ms iteration = 21 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
178132025-11-13T08:09:02.972ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178142025-11-13T08:09:02.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178152025-11-13T08:09:02.977ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178162025-11-13T08:09:02.977ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178172025-11-13T08:09:02.980ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178182025-11-13T08:09:02.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178192025-11-13T08:09:02.983ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178202025-11-13T08:09:02.991ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
178212025-11-13T08:09:02.992ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178222025-11-13T08:09:02.992ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178232025-11-13T08:09:03.000ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178242025-11-13T08:09:03.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178252025-11-13T08:09:03.004ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178262025-11-13T08:09:03.004ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178272025-11-13T08:09:03.007ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178282025-11-13T08:09:03.016ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
178292025-11-13T08:09:03.016ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178302025-11-13T08:09:03.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178312025-11-13T08:09:03.021ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 371.212927ms iteration = 21 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
178322025-11-13T08:09:03.022ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
178332025-11-13T08:09:03.022ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
178342025-11-13T08:09:03.022ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178352025-11-13T08:09:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178362025-11-13T08:09:03.023ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
178372025-11-13T08:09:03.023ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
178382025-11-13T08:09:03.024ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
178392025-11-13T08:09:03.024ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
178402025-11-13T08:09:03.025ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178412025-11-13T08:09:03.025ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178422025-11-13T08:09:03.026ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178432025-11-13T08:09:03.029ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
178442025-11-13T08:09:03.029ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
178452025-11-13T08:09:03.029ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
178462025-11-13T08:09:03.033ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178472025-11-13T08:09:03.033ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178482025-11-13T08:09:03.036ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
178492025-11-13T08:09:03.037ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178502025-11-13T08:09:03.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178512025-11-13T08:09:03.044ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178522025-11-13T08:09:03.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178532025-11-13T08:09:03.048ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
178542025-11-13T08:09:03.048ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 422.21436ms iteration = 31 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
178552025-11-13T08:09:03.050ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178562025-11-13T08:09:03.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178572025-11-13T08:09:03.052ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178582025-11-13T08:09:03.053ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178592025-11-13T08:09:03.056ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178602025-11-13T08:09:03.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178612025-11-13T08:09:03.067ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
178622025-11-13T08:09:03.067ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178632025-11-13T08:09:03.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178642025-11-13T08:09:03.067ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178652025-11-13T08:09:03.069ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178662025-11-13T08:09:03.070ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178672025-11-13T08:09:03.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178682025-11-13T08:09:03.073ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
178692025-11-13T08:09:03.073ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178702025-11-13T08:09:03.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178712025-11-13T08:09:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178722025-11-13T08:09:03.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178732025-11-13T08:09:03.074ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 448.753912ms iteration = 33 status = {"physical_disks_added":0}
178742025-11-13T08:09:03.075ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178752025-11-13T08:09:03.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178762025-11-13T08:09:03.079ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
178772025-11-13T08:09:03.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
178782025-11-13T08:09:03.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
178792025-11-13T08:09:03.080ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
178802025-11-13T08:09:03.082ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
178812025-11-13T08:09:03.082ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178822025-11-13T08:09:03.082ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178832025-11-13T08:09:03.089ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(5588c052-6347-48b0-b6e1-745a625f895b, 0), (484d8991-1076-40d9-9a86-ae1c48054b42, 0), (5cd935de-9a02-47d0-9052-085abf54df67, 0), (d00f4add-2b1d-4d87-8ba8-788f041ad77f, 0)] collection_id = 5588c052-6347-48b0-b6e1-745a625f895b
178842025-11-13T08:09:03.089ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
178852025-11-13T08:09:03.089ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
178862025-11-13T08:09:03.098ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = DeployNexusRecords step id = Ensure
178872025-11-13T08:09:03.098ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178882025-11-13T08:09:03.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178892025-11-13T08:09:03.099ZDEBGnexus (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
178902025-11-13T08:09:03.099ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
178912025-11-13T08:09:03.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
178922025-11-13T08:09:03.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178932025-11-13T08:09:03.103ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
178942025-11-13T08:09:03.105ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
178952025-11-13T08:09:03.105ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
178962025-11-13T08:09:03.105ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
178972025-11-13T08:09:03.105ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
178982025-11-13T08:09:03.106ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "4990a52c-615f-4fed-b162-69d18a40af27", "content-length": "78", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
178992025-11-13T08:09:03.106ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
179002025-11-13T08:09:03.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
179012025-11-13T08:09:03.106ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
179022025-11-13T08:09:03.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
179032025-11-13T08:09:03.107ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
179042025-11-13T08:09:03.107ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
179052025-11-13T08:09:03.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
179062025-11-13T08:09:03.107ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
179072025-11-13T08:09:03.107ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
179082025-11-13T08:09:03.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
179092025-11-13T08:09:03.111ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
179102025-11-13T08:09:03.111ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
179112025-11-13T08:09:03.111ZDEBGnexus (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
179122025-11-13T08:09:03.111ZDEBGnexus (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
179132025-11-13T08:09:03.111ZDEBGnexus (service-firewall-rule-progator): resolved 3 rules for sleds background_task = service_firewall_rule_propagation
179142025-11-13T08:09:03.112ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
179152025-11-13T08:09:03.116ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
179162025-11-13T08:09:03.116ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
179172025-11-13T08:09:03.116ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
179182025-11-13T08:09:03.116ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
179192025-11-13T08:09:03.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
179202025-11-13T08:09:03.116ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
179212025-11-13T08:09:03.116ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
179222025-11-13T08:09:03.118ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
179232025-11-13T08:09:03.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
179242025-11-13T08:09:03.120ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
179252025-11-13T08:09:03.120ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
179262025-11-13T08:09:03.120ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
179272025-11-13T08:09:03.123ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
179282025-11-13T08:09:03.123ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
179292025-11-13T08:09:03.124ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4be5e55-af84-409e-ae41-061dc5a6233d", "content-length": "81", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
179302025-11-13T08:09:03.128ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
179312025-11-13T08:09:03.131ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 524.187875ms iteration = 21 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
179322025-11-13T08:09:03.153ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
179332025-11-13T08:09:03.153ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
179342025-11-13T08:09:03.154ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7a03289-b87b-4d96-aa05-5c6517ee4d9b", "content-length": "26", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
179352025-11-13T08:09:03.154ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
179362025-11-13T08:09:03.157ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
179372025-11-13T08:09:03.163ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179382025-11-13T08:09:03.164ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179392025-11-13T08:09:03.168ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
179402025-11-13T08:09:03.168ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
179412025-11-13T08:09:03.168ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f81c69a2-da42-4192-87b3-633ccc1ec3b5", "content-length": "26", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
179422025-11-13T08:09:03.168ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
179432025-11-13T08:09:03.171ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
179442025-11-13T08:09:03.172ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179452025-11-13T08:09:03.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179462025-11-13T08:09:03.176ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179472025-11-13T08:09:03.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
179482025-11-13T08:09:03.182ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179492025-11-13T08:09:03.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179502025-11-13T08:09:03.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179512025-11-13T08:09:03.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179522025-11-13T08:09:03.195ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179532025-11-13T08:09:03.196ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
179542025-11-13T08:09:03.196ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
179552025-11-13T08:09:03.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179562025-11-13T08:09:03.197ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad710292-19a7-4820-8c89-b47f3e8e52a4", "content-length": "81", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
179572025-11-13T08:09:03.201ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
179582025-11-13T08:09:03.201ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179592025-11-13T08:09:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179602025-11-13T08:09:03.202ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179612025-11-13T08:09:03.203ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
179622025-11-13T08:09:03.203ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
179632025-11-13T08:09:03.204ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "d07e6315-69e0-4d61-b16d-710baf17fce0", "content-length": "81", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
179642025-11-13T08:09:03.205ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
179652025-11-13T08:09:03.206ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
179662025-11-13T08:09:03.206ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
179672025-11-13T08:09:03.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
179682025-11-13T08:09:03.209ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 559.180534ms iteration = 21 status = {"disabled":false,"errors":[],"sps":[]}
179692025-11-13T08:09:03.209ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179702025-11-13T08:09:03.212ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
179712025-11-13T08:09:03.212ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
179722025-11-13T08:09:03.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
179732025-11-13T08:09:03.244ZDEBGnexus (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
179742025-11-13T08:09:03.244ZDEBGnexus (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
179752025-11-13T08:09:03.282ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = SledAgent step id = Ensure
179762025-11-13T08:09:03.285ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179772025-11-13T08:09:03.290ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
179782025-11-13T08:09:03.290ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179792025-11-13T08:09:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179802025-11-13T08:09:03.294ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179812025-11-13T08:09:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179822025-11-13T08:09:03.296ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179832025-11-13T08:09:03.301ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
179842025-11-13T08:09:03.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179852025-11-13T08:09:03.301ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179862025-11-13T08:09:03.303ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
179872025-11-13T08:09:03.303ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 659.855293ms iteration = 11 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
179882025-11-13T08:09:03.307ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179892025-11-13T08:09:03.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179902025-11-13T08:09:03.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179912025-11-13T08:09:03.315ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
179922025-11-13T08:09:03.315ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179932025-11-13T08:09:03.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179942025-11-13T08:09:03.318ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179952025-11-13T08:09:03.318ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
179962025-11-13T08:09:03.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
179972025-11-13T08:09:03.324ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
179982025-11-13T08:09:03.324ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
179992025-11-13T08:09:03.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180002025-11-13T08:09:03.349ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
180012025-11-13T08:09:03.349ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
180022025-11-13T08:09:03.350ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1db2c3e-1893-4c8f-974f-1ca11bcf574b", "content-length": "372", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
180032025-11-13T08:09:03.350ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180042025-11-13T08:09:03.350ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180052025-11-13T08:09:03.355ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
180062025-11-13T08:09:03.358ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180072025-11-13T08:09:03.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180082025-11-13T08:09:03.362ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180092025-11-13T08:09:03.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180102025-11-13T08:09:03.364ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180112025-11-13T08:09:03.369ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
180122025-11-13T08:09:03.369ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180132025-11-13T08:09:03.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180142025-11-13T08:09:03.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180152025-11-13T08:09:03.376ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
180162025-11-13T08:09:03.376ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180172025-11-13T08:09:03.376ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180182025-11-13T08:09:03.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180192025-11-13T08:09:03.379ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180202025-11-13T08:09:03.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180212025-11-13T08:09:03.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180222025-11-13T08:09:03.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180232025-11-13T08:09:03.387ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
180242025-11-13T08:09:03.387ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180252025-11-13T08:09:03.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180262025-11-13T08:09:03.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180272025-11-13T08:09:03.394ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
180282025-11-13T08:09:03.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180292025-11-13T08:09:03.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180302025-11-13T08:09:03.398ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180312025-11-13T08:09:03.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180322025-11-13T08:09:03.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180332025-11-13T08:09:03.399ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 5588c052-6347-48b0-b6e1-745a625f895b file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
180342025-11-13T08:09:03.399ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
180352025-11-13T08:09:03.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180362025-11-13T08:09:03.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
180372025-11-13T08:09:03.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180382025-11-13T08:09:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
180392025-11-13T08:09:03.418ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
180402025-11-13T08:09:03.418ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
180412025-11-13T08:09:03.419ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "e483caae-81e3-4d40-bcc7-582734aa3027", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
180422025-11-13T08:09:03.419ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180432025-11-13T08:09:03.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180442025-11-13T08:09:03.422ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180452025-11-13T08:09:03.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180462025-11-13T08:09:03.428ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
180472025-11-13T08:09:03.428ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
180482025-11-13T08:09:03.433ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
180492025-11-13T08:09:03.433ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
180502025-11-13T08:09:03.436ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "9b47b62b-ff40-4d49-96b6-48cc08fefea7", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
180512025-11-13T08:09:03.438ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 793.791205ms iteration = 31 status = {}
180522025-11-13T08:09:03.463ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180532025-11-13T08:09:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180542025-11-13T08:09:03.464ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180552025-11-13T08:09:03.464ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180562025-11-13T08:09:03.466ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180572025-11-13T08:09:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
180582025-11-13T08:09:03.488ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6551, tv_nsec: 840385211 } })
180592025-11-13T08:09:03.512ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
180602025-11-13T08:09:03.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
180612025-11-13T08:09:03.514ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
180622025-11-13T08:09:03.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
180632025-11-13T08:09:03.519ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180642025-11-13T08:09:03.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180652025-11-13T08:09:03.524ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180662025-11-13T08:09:03.524ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180672025-11-13T08:09:03.526ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180682025-11-13T08:09:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
180692025-11-13T08:09:03.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180702025-11-13T08:09:03.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180712025-11-13T08:09:03.531ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180722025-11-13T08:09:03.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180732025-11-13T08:09:03.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180742025-11-13T08:09:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
180752025-11-13T08:09:03.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180762025-11-13T08:09:03.537ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180772025-11-13T08:09:03.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180782025-11-13T08:09:03.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180792025-11-13T08:09:03.540ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180802025-11-13T08:09:03.545ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
180812025-11-13T08:09:03.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180822025-11-13T08:09:03.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180832025-11-13T08:09:03.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180842025-11-13T08:09:03.548ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180852025-11-13T08:09:03.563ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(5cd935de-9a02-47d0-9052-085abf54df67, 0), (d00f4add-2b1d-4d87-8ba8-788f041ad77f, 0), (1fd1eb26-3b73-4aab-81aa-76414004f934, 0)]
180862025-11-13T08:09:03.563ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6551, tv_nsec: 915914786 } })
180872025-11-13T08:09:03.566ZDEBGnexus (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180882025-11-13T08:09:03.570ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_firewall_rule_propagation resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
180892025-11-13T08:09:03.570ZDEBGnexus (service-firewall-rule-progator): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = service_firewall_rule_propagation body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
180902025-11-13T08:09:03.571ZDEBGnexus (service-firewall-rule-progator): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = service_firewall_rule_propagation result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "7f62ddc4-985d-4df3-9a07-e9168ab3fdc6", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
180912025-11-13T08:09:03.573ZINFOnexus (service-firewall-rule-progator): updated firewall rules on 1 sleds background_task = service_firewall_rule_propagation file = nexus/networking/src/firewall_rules.rs:491
180922025-11-13T08:09:03.573ZDEBGnexus (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 931.114026ms
180932025-11-13T08:09:03.573ZDEBGnexus (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 931.131726ms iteration = 3 status = {}
180942025-11-13T08:09:03.586ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
180952025-11-13T08:09:03.588ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
180962025-11-13T08:09:03.589ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
180972025-11-13T08:09:03.589ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
180982025-11-13T08:09:03.589ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
180992025-11-13T08:09:03.598ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "5e69dbc8-935f-4473-9549-cdc91deb03ff", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
181002025-11-13T08:09:03.599ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = FirewallRules step id = Ensure
181012025-11-13T08:09:03.599ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181022025-11-13T08:09:03.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181032025-11-13T08:09:03.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181042025-11-13T08:09:03.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181052025-11-13T08:09:03.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181062025-11-13T08:09:03.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181072025-11-13T08:09:03.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181082025-11-13T08:09:03.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181092025-11-13T08:09:03.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181102025-11-13T08:09:03.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181112025-11-13T08:09:03.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181122025-11-13T08:09:03.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181132025-11-13T08:09:03.613ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181142025-11-13T08:09:03.618ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
181152025-11-13T08:09:03.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181162025-11-13T08:09:03.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181172025-11-13T08:09:03.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181182025-11-13T08:09:03.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181192025-11-13T08:09:03.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181202025-11-13T08:09:03.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181212025-11-13T08:09:03.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181222025-11-13T08:09:03.626ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
181232025-11-13T08:09:03.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181242025-11-13T08:09:03.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181252025-11-13T08:09:03.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181262025-11-13T08:09:03.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181272025-11-13T08:09:03.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181282025-11-13T08:09:03.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181292025-11-13T08:09:03.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181302025-11-13T08:09:03.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181312025-11-13T08:09:03.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181322025-11-13T08:09:03.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181332025-11-13T08:09:03.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181342025-11-13T08:09:03.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181352025-11-13T08:09:03.636ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
181362025-11-13T08:09:03.636ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
181372025-11-13T08:09:03.636ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = deeeb3de-6188-464a-8c0d-a3578498da64 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
181382025-11-13T08:09:03.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
181392025-11-13T08:09:03.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
181402025-11-13T08:09:03.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181412025-11-13T08:09:03.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181422025-11-13T08:09:03.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181432025-11-13T08:09:03.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181442025-11-13T08:09:03.639ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
181452025-11-13T08:09:03.640ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "bd249c48-153c-4d20-a5bd-357c07113099", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
181462025-11-13T08:09:03.640ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
181472025-11-13T08:09:03.641ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c5bbf451-f6be-4587-9358-8240abfbcf9a", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
181482025-11-13T08:09:03.641ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
181492025-11-13T08:09:03.642ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
181502025-11-13T08:09:03.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
181512025-11-13T08:09:03.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
181522025-11-13T08:09:03.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
181532025-11-13T08:09:03.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181542025-11-13T08:09:03.644ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "c8664024-a70b-4ba5-a330-8596f479aadb", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
181552025-11-13T08:09:03.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181562025-11-13T08:09:03.644ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
181572025-11-13T08:09:03.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181582025-11-13T08:09:03.649ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
181592025-11-13T08:09:03.649ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 1.042373453s iteration = 13 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
181602025-11-13T08:09:03.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181612025-11-13T08:09:03.650ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
181622025-11-13T08:09:03.650ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
181632025-11-13T08:09:03.650ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 1.031257464s iteration = 25 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
181642025-11-13T08:09:03.651ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
181652025-11-13T08:09:03.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181662025-11-13T08:09:03.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
181672025-11-13T08:09:03.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181682025-11-13T08:09:03.654ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181692025-11-13T08:09:03.655ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181702025-11-13T08:09:03.655ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
181712025-11-13T08:09:03.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181722025-11-13T08:09:03.656ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "3174f629-93a9-49b1-a60a-c190b48e445e", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
181732025-11-13T08:09:03.656ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
181742025-11-13T08:09:03.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
181752025-11-13T08:09:03.657ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
181762025-11-13T08:09:03.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
181772025-11-13T08:09:03.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181782025-11-13T08:09:03.659ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9a4b4c4c-da43-4ade-851d-5448f7b45486", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
181792025-11-13T08:09:03.659ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
181802025-11-13T08:09:03.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181812025-11-13T08:09:03.660ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
181822025-11-13T08:09:03.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
181832025-11-13T08:09:03.661ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181842025-11-13T08:09:03.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181852025-11-13T08:09:03.662ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181862025-11-13T08:09:03.663ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181872025-11-13T08:09:03.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181882025-11-13T08:09:03.664ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
181892025-11-13T08:09:03.664ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 1.056946709s iteration = 13 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
181902025-11-13T08:09:03.664ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
181912025-11-13T08:09:03.665ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
181922025-11-13T08:09:03.665ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
181932025-11-13T08:09:03.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
181942025-11-13T08:09:03.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
181952025-11-13T08:09:03.670ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
181962025-11-13T08:09:03.670ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181972025-11-13T08:09:03.670ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
181982025-11-13T08:09:03.672ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
181992025-11-13T08:09:03.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182002025-11-13T08:09:03.673ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "c2843ce1-23b8-42cc-a8f4-08d315c0cad8", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182012025-11-13T08:09:03.673ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
182022025-11-13T08:09:03.674ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182032025-11-13T08:09:03.675ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182042025-11-13T08:09:03.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182052025-11-13T08:09:03.676ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
182062025-11-13T08:09:03.676ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 1.026300757s iteration = 11 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
182072025-11-13T08:09:03.676ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 32 reason = Dependency
182082025-11-13T08:09:03.676ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
182092025-11-13T08:09:03.676ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
182102025-11-13T08:09:03.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
182112025-11-13T08:09:03.677ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 1.070462025s iteration = 21 status = {}
182122025-11-13T08:09:03.678ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182132025-11-13T08:09:03.681ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
182142025-11-13T08:09:03.681ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
182152025-11-13T08:09:03.682ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
182162025-11-13T08:09:03.682ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
182172025-11-13T08:09:03.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
182182025-11-13T08:09:03.684ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182192025-11-13T08:09:03.685ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182202025-11-13T08:09:03.686ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
182212025-11-13T08:09:03.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182222025-11-13T08:09:03.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182232025-11-13T08:09:03.689ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
182242025-11-13T08:09:03.689ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
182252025-11-13T08:09:03.689ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
182262025-11-13T08:09:03.689ZDEBGnexus (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
182272025-11-13T08:09:03.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
182282025-11-13T08:09:03.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
182292025-11-13T08:09:03.690ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/artifacts-config
182302025-11-13T08:09:03.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182312025-11-13T08:09:03.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182322025-11-13T08:09:03.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182332025-11-13T08:09:03.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182342025-11-13T08:09:03.695ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182352025-11-13T08:09:03.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182362025-11-13T08:09:03.696ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts-config", status: 204, headers: {"x-request-id": "ccc298dd-5509-4ac6-9f4d-090ebeba3429", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182372025-11-13T08:09:03.696ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/artifacts
182382025-11-13T08:09:03.696ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = tuf_artifact_replication result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "d29ab7e4-955d-482a-80f0-fca26f8d73c1", "content-length": "26", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182392025-11-13T08:09:03.696ZINFOnexus (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication file = nexus/src/app/background/tasks/tuf_artifact_replication.rs:679 sled = http://[fd00:1122:3344:101::1]:12345
182402025-11-13T08:09:03.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182412025-11-13T08:09:03.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182422025-11-13T08:09:03.698ZDEBGnexus (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 1.04984247s iteration = 3 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":2,"put_config_err":0,"put_config_ok":2,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"370ec2da-8f7f-4861-9b77-369c2bb1c86d","time":"2025-11-13T08:04:03.190504056Z"},{"operation":"list","target_sled":"370ec2da-8f7f-4861-9b77-369c2bb1c86d","time":"2025-11-13T08:04:03.190815752Z"},{"generation":1,"operation":"put_config","target_sled":"370ec2da-8f7f-4861-9b77-369c2bb1c86d","time":"2025-11-13T08:09:03.696322280Z"},{"operation":"list","target_sled":"370ec2da-8f7f-4861-9b77-369c2bb1c86d","time":"2025-11-13T08:09:03.696639217Z"}]}
182432025-11-13T08:09:03.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182442025-11-13T08:09:03.700ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182452025-11-13T08:09:03.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182462025-11-13T08:09:03.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182472025-11-13T08:09:03.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182482025-11-13T08:09:03.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182492025-11-13T08:09:03.704ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182502025-11-13T08:09:03.709ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
182512025-11-13T08:09:03.709ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182522025-11-13T08:09:03.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182532025-11-13T08:09:03.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182542025-11-13T08:09:03.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182552025-11-13T08:09:03.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182562025-11-13T08:09:03.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182572025-11-13T08:09:03.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182582025-11-13T08:09:03.718ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
182592025-11-13T08:09:03.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182602025-11-13T08:09:03.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182612025-11-13T08:09:03.721ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
182622025-11-13T08:09:03.721ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e10ae98-e02e-4749-b05c-7e1718fca42c", "content-length": "691", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182632025-11-13T08:09:03.721ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
182642025-11-13T08:09:03.721ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
182652025-11-13T08:09:03.721ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
182662025-11-13T08:09:03.723ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 1.079520923s iteration = 21 status = {}
182672025-11-13T08:09:03.726ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
182682025-11-13T08:09:03.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182692025-11-13T08:09:03.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182702025-11-13T08:09:03.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
182712025-11-13T08:09:03.730ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
182722025-11-13T08:09:03.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
182732025-11-13T08:09:03.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
182742025-11-13T08:09:03.732ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
182752025-11-13T08:09:03.732ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
182762025-11-13T08:09:03.732ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
182772025-11-13T08:09:03.736ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6552, tv_nsec: 88519380 } })
182782025-11-13T08:09:03.737ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
182792025-11-13T08:09:03.737ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
182802025-11-13T08:09:03.737ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
182812025-11-13T08:09:03.737ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7b91365-97a6-42e8-8c6f-2cca3dadd7b5", "content-length": "476", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182822025-11-13T08:09:03.737ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
182832025-11-13T08:09:03.737ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c3a3f3b-b151-447b-9109-b7b3b62ac763", "content-length": "734", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182842025-11-13T08:09:03.738ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
182852025-11-13T08:09:03.738ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37fdfd0d-0f2d-4814-befc-12392088e91e", "content-length": "111", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182862025-11-13T08:09:03.738ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
182872025-11-13T08:09:03.739ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bac42e7-b3ba-415d-97eb-0c05cb93198b", "content-length": "111", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182882025-11-13T08:09:03.739ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
182892025-11-13T08:09:03.739ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef54ff9f-87b7-487c-b08a-cd7485b1872e", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182902025-11-13T08:09:03.739ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
182912025-11-13T08:09:03.739ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dc54b80-70e1-422b-aee2-a040fce9db5b", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182922025-11-13T08:09:03.739ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
182932025-11-13T08:09:03.740ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "15a01572-c450-4fbb-9fe7-563b4e799980", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182942025-11-13T08:09:03.740ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
182952025-11-13T08:09:03.740ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc7c910a-5216-4d46-b5e3-9a19aadfdf60", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182962025-11-13T08:09:03.740ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
182972025-11-13T08:09:03.740ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db382af4-9a7e-4a30-b244-daa39761cfe0", "content-length": "702", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
182982025-11-13T08:09:03.741ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
182992025-11-13T08:09:03.741ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a3785df-54d9-4371-b36c-9fdaff429a57", "content-length": "727", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183002025-11-13T08:09:03.741ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
183012025-11-13T08:09:03.741ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "837e4f91-5c9d-4cfd-b16c-b1fd26e1a6b0", "content-length": "729", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183022025-11-13T08:09:03.741ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
183032025-11-13T08:09:03.741ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8621e8dc-b741-4fb7-8802-0f7758970c44", "content-length": "728", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183042025-11-13T08:09:03.741ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
183052025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6c2fe79-f695-4a83-bf18-96818828e938", "content-length": "722", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183062025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
183072025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "52d4c5ed-26d9-411a-a29b-6710feb410c5", "content-length": "10", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183082025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
183092025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e35fc3a9-f716-4379-9b7c-dfeb7c025366", "content-length": "144", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183102025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
183112025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0de8cd2-9094-4e27-9f40-ea1f55c546be", "content-length": "147", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183122025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
183132025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db53748b-ed0a-4e43-992b-cfaa7658590c", "content-length": "109", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183142025-11-13T08:09:03.742ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
183152025-11-13T08:09:03.743ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "773f7f54-892d-4796-a685-7d2fccad27bd", "content-length": "109", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183162025-11-13T08:09:03.743ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
183172025-11-13T08:09:03.743ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "66991a33-26fb-44cf-bb94-92932b1dd2b0", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183182025-11-13T08:09:03.743ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
183192025-11-13T08:09:03.743ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14f31a43-9704-47c5-86b0-21b9990b72b3", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183202025-11-13T08:09:03.743ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
183212025-11-13T08:09:03.744ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7602a69b-429d-4b4f-a6df-f31fd50248d8", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183222025-11-13T08:09:03.744ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
183232025-11-13T08:09:03.744ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b151c6b8-857b-4cfb-b839-b36a50f5630a", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183242025-11-13T08:09:03.744ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
183252025-11-13T08:09:03.744ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "56fa6afe-98d7-4fae-93e1-f720f6748636", "content-length": "702", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183262025-11-13T08:09:03.744ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
183272025-11-13T08:09:03.745ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb423d1b-0451-44f5-afae-0b01b6a70499", "content-length": "727", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183282025-11-13T08:09:03.745ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
183292025-11-13T08:09:03.745ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2c787da-571b-4c3e-b88e-0ca53e0b7a5b", "content-length": "729", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183302025-11-13T08:09:03.745ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
183312025-11-13T08:09:03.745ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2db85e53-78a1-4b5a-aeb0-472354824f2e", "content-length": "728", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183322025-11-13T08:09:03.745ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
183332025-11-13T08:09:03.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
183342025-11-13T08:09:03.747ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183352025-11-13T08:09:03.751ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
183362025-11-13T08:09:03.751ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
183372025-11-13T08:09:03.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
183382025-11-13T08:09:03.753ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183392025-11-13T08:09:03.756ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
183402025-11-13T08:09:03.756ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
183412025-11-13T08:09:03.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
183422025-11-13T08:09:03.771ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
183432025-11-13T08:09:03.771ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
183442025-11-13T08:09:03.772ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "328bab3f-2481-4763-a303-da1a1bb67032", "content-length": "372", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183452025-11-13T08:09:03.772ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
183462025-11-13T08:09:03.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
183472025-11-13T08:09:03.794ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
183482025-11-13T08:09:03.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
183492025-11-13T08:09:03.797ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
183502025-11-13T08:09:03.797ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
183512025-11-13T08:09:03.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
183522025-11-13T08:09:03.805ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
183532025-11-13T08:09:03.805ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
183542025-11-13T08:09:03.813ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
183552025-11-13T08:09:03.813ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
183562025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
183572025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
183582025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
183592025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
183602025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
183612025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
183622025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
183632025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
183642025-11-13T08:09:03.814ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
183652025-11-13T08:09:03.814ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
183662025-11-13T08:09:03.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
183672025-11-13T08:09:03.817ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
183682025-11-13T08:09:03.817ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
183692025-11-13T08:09:03.836ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
183702025-11-13T08:09:03.836ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
183712025-11-13T08:09:03.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183722025-11-13T08:09:03.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183732025-11-13T08:09:03.840ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "f06743eb-6860-46a0-b09a-b4f817fddce9", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183742025-11-13T08:09:03.840ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
183752025-11-13T08:09:03.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
183762025-11-13T08:09:03.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183772025-11-13T08:09:03.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183782025-11-13T08:09:03.842ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183792025-11-13T08:09:03.843ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
183802025-11-13T08:09:03.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
183812025-11-13T08:09:03.846ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
183822025-11-13T08:09:03.846ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
183832025-11-13T08:09:03.847ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "6507a1b9-6220-48fc-bc95-4e88020ed2a9", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183842025-11-13T08:09:03.848ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
183852025-11-13T08:09:03.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
183862025-11-13T08:09:03.850ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
183872025-11-13T08:09:03.850ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = Dns step id = Ensure
183882025-11-13T08:09:03.850ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
183892025-11-13T08:09:03.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183902025-11-13T08:09:03.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183912025-11-13T08:09:03.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
183922025-11-13T08:09:03.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
183932025-11-13T08:09:03.852ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
183942025-11-13T08:09:03.852ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
183952025-11-13T08:09:03.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
183962025-11-13T08:09:03.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
183972025-11-13T08:09:03.858ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "d55c8b27-a1d0-4ec8-8257-5705960d8237", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
183982025-11-13T08:09:03.859ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
183992025-11-13T08:09:03.859ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
184002025-11-13T08:09:03.859ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
184012025-11-13T08:09:03.859ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184022025-11-13T08:09:03.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184032025-11-13T08:09:03.860ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 184.573019ms iteration = 32 status = {}
184042025-11-13T08:09:03.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184052025-11-13T08:09:03.865ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
184062025-11-13T08:09:03.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184072025-11-13T08:09:03.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184082025-11-13T08:09:03.876ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
184092025-11-13T08:09:03.876ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
184102025-11-13T08:09:03.904ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
184112025-11-13T08:09:03.904ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
184122025-11-13T08:09:03.904ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184132025-11-13T08:09:03.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184142025-11-13T08:09:03.907ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184152025-11-13T08:09:03.910ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
184162025-11-13T08:09:03.910ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184172025-11-13T08:09:03.910ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184182025-11-13T08:09:03.913ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
184192025-11-13T08:09:03.913ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
184202025-11-13T08:09:03.913ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
184212025-11-13T08:09:03.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184222025-11-13T08:09:03.913ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184232025-11-13T08:09:03.914ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184242025-11-13T08:09:03.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184252025-11-13T08:09:03.918ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184262025-11-13T08:09:03.922ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
184272025-11-13T08:09:03.923ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
184282025-11-13T08:09:03.923ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
184292025-11-13T08:09:03.923ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
184302025-11-13T08:09:03.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184312025-11-13T08:09:03.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184322025-11-13T08:09:03.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184332025-11-13T08:09:03.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
184342025-11-13T08:09:03.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184352025-11-13T08:09:03.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184362025-11-13T08:09:03.932ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
184372025-11-13T08:09:03.932ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
184382025-11-13T08:09:03.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184392025-11-13T08:09:03.932ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184402025-11-13T08:09:03.934ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184412025-11-13T08:09:03.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
184422025-11-13T08:09:03.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184432025-11-13T08:09:03.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184442025-11-13T08:09:03.939ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
184452025-11-13T08:09:03.939ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
184462025-11-13T08:09:03.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184472025-11-13T08:09:03.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184482025-11-13T08:09:03.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184492025-11-13T08:09:03.943ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
184502025-11-13T08:09:03.943ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184512025-11-13T08:09:03.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184522025-11-13T08:09:03.946ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184532025-11-13T08:09:03.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184542025-11-13T08:09:03.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
184552025-11-13T08:09:03.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
184562025-11-13T08:09:03.952ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "f439d496-e575-4734-8f0a-c77e1b458029", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184572025-11-13T08:09:03.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
184582025-11-13T08:09:03.955ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
184592025-11-13T08:09:03.955ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
184602025-11-13T08:09:03.955ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
184612025-11-13T08:09:03.955ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
184622025-11-13T08:09:03.955ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
184632025-11-13T08:09:03.955ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
184642025-11-13T08:09:03.955ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
184652025-11-13T08:09:03.955ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
184662025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a0092c4-e99a-4f8e-8acc-d8056d8a5e2f", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184672025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
184682025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "615d2b25-fbbc-474f-8de4-6a572e723d4c", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184692025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
184702025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c756606-535d-42bd-82f1-7f9d67544b97", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184712025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c494b7e-e5e1-47db-b4a7-6954d19a2dac", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184722025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
184732025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
184742025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "49fdeab3-bc6e-4d4a-b1df-84d5559d4265", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184752025-11-13T08:09:03.956ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
184762025-11-13T08:09:03.957ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
184772025-11-13T08:09:03.960ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
184782025-11-13T08:09:03.960ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
184792025-11-13T08:09:03.960ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = OmicronZones step id = Cleanup
184802025-11-13T08:09:03.960ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = OmicronZones step id = Cleanup
184812025-11-13T08:09:03.960ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = PhysicalDisks step id = Cleanup
184822025-11-13T08:09:03.960ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = Clickhouse step id = Ensure
184832025-11-13T08:09:03.960ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = Clickhouse step id = Ensure
184842025-11-13T08:09:03.961ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "ca0fb4fb-5036-416f-809b-171e289851ed", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184852025-11-13T08:09:03.961ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
184862025-11-13T08:09:03.961ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
184872025-11-13T08:09:03.961ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bbe21fb-0d08-4bb4-aabc-6377fa05b0c7", "content-length": "682", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184882025-11-13T08:09:03.961ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
184892025-11-13T08:09:03.961ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
184902025-11-13T08:09:03.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
184912025-11-13T08:09:03.965ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
184922025-11-13T08:09:03.969ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
184932025-11-13T08:09:03.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
184942025-11-13T08:09:03.971ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
184952025-11-13T08:09:03.971ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
184962025-11-13T08:09:03.980ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.336435794s iteration = 21 status = {}
184972025-11-13T08:09:03.994ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
184982025-11-13T08:09:03.996ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "66aafddc-0105-49f8-8f04-44321c6826fa", "content-length": "412723", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
184992025-11-13T08:09:03.997ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad569435-afa6-4582-95fc-2b2a5669216c", "content-length": "406041", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
185002025-11-13T08:09:03.998ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fe57ea9-1075-423b-b7d8-5567bee0b7cb", "content-length": "427525", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
185012025-11-13T08:09:03.999ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "841c9170-70f8-4eae-b283-4f42d21fe4b8", "content-length": "405389", "date": "Thu, 13 Nov 2025 08:09:03 GMT"} })
185022025-11-13T08:09:04.000ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "9223eff1-cc55-46ea-a7f8-0e92ff066ae5", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
185032025-11-13T08:09:04.002ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a866d59-883c-49b8-aa7f-826949ab5296", "content-length": "405534", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
185042025-11-13T08:09:04.003ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = SupportBundles step id = Cleanup
185052025-11-13T08:09:04.003ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
185062025-11-13T08:09:04.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
185072025-11-13T08:09:04.005ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
185082025-11-13T08:09:04.005ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
185092025-11-13T08:09:04.005ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
185102025-11-13T08:09:04.005ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
185112025-11-13T08:09:04.005ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
185122025-11-13T08:09:04.005ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185132025-11-13T08:09:04.006ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
185142025-11-13T08:09:04.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
185152025-11-13T08:09:04.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
185162025-11-13T08:09:04.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
185172025-11-13T08:09:04.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
185182025-11-13T08:09:04.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185192025-11-13T08:09:04.013ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
185202025-11-13T08:09:04.037ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
185212025-11-13T08:09:04.037ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
185222025-11-13T08:09:04.040ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "44840d47-26e4-4df9-9f65-cfb74b6506bc", "content-length": "124", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
185232025-11-13T08:09:04.041ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
185242025-11-13T08:09:04.052ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = OmicronZones step id = Cleanup
185252025-11-13T08:09:04.052ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
185262025-11-13T08:09:04.052ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
185272025-11-13T08:09:04.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
185282025-11-13T08:09:04.052ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
185292025-11-13T08:09:04.054ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185302025-11-13T08:09:04.056ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
185312025-11-13T08:09:04.056ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
185322025-11-13T08:09:04.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
185332025-11-13T08:09:04.093ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
185342025-11-13T08:09:04.093ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
185352025-11-13T08:09:04.093ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = Cockroach step id = Ensure
185362025-11-13T08:09:04.093ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
185372025-11-13T08:09:04.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
185382025-11-13T08:09:04.130ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
185392025-11-13T08:09:04.130ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = fe785789-1e3e-4428-b365-cc342884bb36 step component = MgsUpdates step id = Ensure
185402025-11-13T08:09:04.130ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 32 reason = Dependency
185412025-11-13T08:09:04.130ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
185422025-11-13T08:09:04.131ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.523241177s iteration = 12 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"814060db-6808-4846-87cd-afadeac11617","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3440,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":109546784,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":109546734,"secs":0}},"event_index":1,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":109560454,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":381233515,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":381233465,"secs":0}},"event_index":2,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":490807919,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":183376815,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":183376766,"secs":0}},"event_index":3,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":674205045,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":317724707,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":317724656,"secs":0}},"event_index":4,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":991962612,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":250304361,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":250304301,"secs":0}},"event_index":5,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":242279803,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":110010263,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":110010213,"secs":0}},"event_index":6,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":352297846,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4170,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4130,"secs":0}},"event_index":7,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":352315396,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2470,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2430,"secs":0}},"event_index":8,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":352320256,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3580,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3550,"secs":0}},"event_index":9,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":352325216,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1760,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1720,"secs":0}},"event_index":10,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":352328596,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":42914764,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":42914714,"secs":0}},"event_index":11,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":395244360,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49237878,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":49237848,"secs":0}},"event_index":12,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":444496148,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":41441905,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":41441865,"secs":0}},"event_index":13,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":485960444,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36920215,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":36920166,"secs":0}},"event_index":14,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":522889559,"secs":1}},{"data":{"attempt_elapsed":{"nanos":5650,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":5610,"secs":0}},"event_index":15,"execution_id":"814060db-6808-4846-87cd-afadeac11617","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":522906950,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
185432025-11-13T08:09:04.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
185442025-11-13T08:09:04.132ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185452025-11-13T08:09:04.135ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
185462025-11-13T08:09:04.135ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
185472025-11-13T08:09:04.135ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
185482025-11-13T08:09:04.139ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
185492025-11-13T08:09:04.139ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
185502025-11-13T08:09:04.139ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "593ec55c-a566-4d1e-97a9-fcd0a983f8a2", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
185512025-11-13T08:09:04.140ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
185522025-11-13T08:09:04.141ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
185532025-11-13T08:09:04.141ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
185542025-11-13T08:09:04.141ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdc71ff0-2de3-426d-bffe-925d099a8bab", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
185552025-11-13T08:09:04.142ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
185562025-11-13T08:09:04.144ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
185572025-11-13T08:09:04.144ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
185582025-11-13T08:09:04.144ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "d40f1617-934e-49f1-a012-048bbe4a9e4f", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
185592025-11-13T08:09:04.144ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
185602025-11-13T08:09:04.147ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
185612025-11-13T08:09:04.147ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
185622025-11-13T08:09:04.149ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
185632025-11-13T08:09:04.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
185642025-11-13T08:09:04.150ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185652025-11-13T08:09:04.153ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
185662025-11-13T08:09:04.153ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
185672025-11-13T08:09:04.153ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
185682025-11-13T08:09:04.178ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
185692025-11-13T08:09:04.179ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
185702025-11-13T08:09:04.180ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185712025-11-13T08:09:04.182ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
185722025-11-13T08:09:04.182ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
185732025-11-13T08:09:04.183ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
185742025-11-13T08:09:04.185ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
185752025-11-13T08:09:04.185ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 54.230186ms iteration = 32 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
185762025-11-13T08:09:04.354ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 3cbe92e3-7f28-4466-b3c2-56fa669e6e95 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
185772025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 3cbe92e3-7f28-4466-b3c2-56fa669e6e95 time_started = 2025-11-13 08:09:03.736972 UTC
185782025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 53 reason = Dependency
185792025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
185802025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.743531337s iteration = 12 status = {"collection_id":"3cbe92e3-7f28-4466-b3c2-56fa669e6e95","time_done":"2025-11-13 08:09:04.147116 UTC","time_started":"2025-11-13 08:09:03.736972 UTC"}
185812025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Dependency
185822025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
185832025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
185842025-11-13T08:09:04.354ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
185852025-11-13T08:09:04.358ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185862025-11-13T08:09:04.359ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
185872025-11-13T08:09:04.361ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
185882025-11-13T08:09:04.361ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
185892025-11-13T08:09:04.361ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
185902025-11-13T08:09:04.362ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
185912025-11-13T08:09:04.362ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
185922025-11-13T08:09:04.362ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
185932025-11-13T08:09:04.368ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
185942025-11-13T08:09:04.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
185952025-11-13T08:09:04.407ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(d00f4add-2b1d-4d87-8ba8-788f041ad77f, 0), (1fd1eb26-3b73-4aab-81aa-76414004f934, 0), (b936028d-8e78-4156-af5a-22c410c90fd2, 0), (7bad19a7-d3d3-4161-9b30-7a7b23a70a0e, 0)] collection_id = d00f4add-2b1d-4d87-8ba8-788f041ad77f
185962025-11-13T08:09:04.407ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
185972025-11-13T08:09:04.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
185982025-11-13T08:09:04.423ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 16 reason = Dependency
185992025-11-13T08:09:04.423ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 69.514707ms iteration = 53 status = {"Loaded":{"collection_id":"3cbe92e3-7f28-4466-b3c2-56fa669e6e95","time_loaded":"2025-11-13T08:09:04.354388537Z","time_started":"2025-11-13T08:09:03.736972Z"}}
186002025-11-13T08:09:04.423ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 26 reason = Dependency
186012025-11-13T08:09:04.423ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 34 reason = Dependency
186022025-11-13T08:09:04.423ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
186032025-11-13T08:09:04.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186042025-11-13T08:09:04.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186052025-11-13T08:09:04.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
186062025-11-13T08:09:04.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186072025-11-13T08:09:04.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186082025-11-13T08:09:04.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186092025-11-13T08:09:04.425ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186102025-11-13T08:09:04.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186112025-11-13T08:09:04.428ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
186122025-11-13T08:09:04.428ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186132025-11-13T08:09:04.429ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
186142025-11-13T08:09:04.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186152025-11-13T08:09:04.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186162025-11-13T08:09:04.429ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
186172025-11-13T08:09:04.429ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
186182025-11-13T08:09:04.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186192025-11-13T08:09:04.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
186202025-11-13T08:09:04.430ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186212025-11-13T08:09:04.431ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
186222025-11-13T08:09:04.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
186232025-11-13T08:09:04.433ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
186242025-11-13T08:09:04.433ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186252025-11-13T08:09:04.433ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186262025-11-13T08:09:04.433ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186272025-11-13T08:09:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
186282025-11-13T08:09:04.436ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
186292025-11-13T08:09:04.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
186302025-11-13T08:09:04.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186312025-11-13T08:09:04.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186322025-11-13T08:09:04.480ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186332025-11-13T08:09:04.482ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
186342025-11-13T08:09:04.482ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186352025-11-13T08:09:04.482ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186362025-11-13T08:09:04.483ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186372025-11-13T08:09:04.485ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
186382025-11-13T08:09:04.485ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186392025-11-13T08:09:04.485ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186402025-11-13T08:09:04.486ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 62.382066ms iteration = 34 status = {"physical_disks_added":0}
186412025-11-13T08:09:04.487ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186422025-11-13T08:09:04.490ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
186432025-11-13T08:09:04.490ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186442025-11-13T08:09:04.490ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186452025-11-13T08:09:04.493ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
186462025-11-13T08:09:04.493ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186472025-11-13T08:09:04.494ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186482025-11-13T08:09:04.496ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186492025-11-13T08:09:04.500ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
186502025-11-13T08:09:04.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186512025-11-13T08:09:04.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186522025-11-13T08:09:04.502ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186532025-11-13T08:09:04.504ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
186542025-11-13T08:09:04.504ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
186552025-11-13T08:09:04.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
186562025-11-13T08:09:04.507ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
186572025-11-13T08:09:04.507ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 83.646255ms iteration = 16 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"3cbe92e3-7f28-4466-b3c2-56fa669e6e95","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
186582025-11-13T08:09:04.562ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186592025-11-13T08:09:04.563ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186602025-11-13T08:09:04.564ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186612025-11-13T08:09:04.566ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
186622025-11-13T08:09:04.566ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186632025-11-13T08:09:04.566ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186642025-11-13T08:09:04.568ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186652025-11-13T08:09:04.570ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
186662025-11-13T08:09:04.570ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186672025-11-13T08:09:04.570ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186682025-11-13T08:09:04.574ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186692025-11-13T08:09:04.574ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186702025-11-13T08:09:04.576ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186712025-11-13T08:09:04.576ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186722025-11-13T08:09:04.577ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186732025-11-13T08:09:04.580ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
186742025-11-13T08:09:04.580ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186752025-11-13T08:09:04.580ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186762025-11-13T08:09:04.582ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186772025-11-13T08:09:04.582ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186782025-11-13T08:09:04.584ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186792025-11-13T08:09:04.587ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
186802025-11-13T08:09:04.587ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186812025-11-13T08:09:04.587ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186822025-11-13T08:09:04.588ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186832025-11-13T08:09:04.591ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
186842025-11-13T08:09:04.591ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186852025-11-13T08:09:04.591ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186862025-11-13T08:09:04.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186872025-11-13T08:09:04.593ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186882025-11-13T08:09:04.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186892025-11-13T08:09:04.598ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
186902025-11-13T08:09:04.598ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186912025-11-13T08:09:04.598ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186922025-11-13T08:09:04.601ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186932025-11-13T08:09:04.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186942025-11-13T08:09:04.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
186952025-11-13T08:09:04.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
186962025-11-13T08:09:04.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186972025-11-13T08:09:04.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
186982025-11-13T08:09:04.609ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
186992025-11-13T08:09:04.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187002025-11-13T08:09:04.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187012025-11-13T08:09:04.614ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
187022025-11-13T08:09:04.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187032025-11-13T08:09:04.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187042025-11-13T08:09:04.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187052025-11-13T08:09:04.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187062025-11-13T08:09:04.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187072025-11-13T08:09:04.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187082025-11-13T08:09:04.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187092025-11-13T08:09:04.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
187102025-11-13T08:09:04.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187112025-11-13T08:09:04.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187122025-11-13T08:09:04.639ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 10 transaction = inventory_delete_collection
187132025-11-13T08:09:04.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187142025-11-13T08:09:04.642ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
187152025-11-13T08:09:04.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187162025-11-13T08:09:04.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187172025-11-13T08:09:04.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187182025-11-13T08:09:04.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187192025-11-13T08:09:04.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187202025-11-13T08:09:04.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187212025-11-13T08:09:04.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187222025-11-13T08:09:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
187232025-11-13T08:09:04.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187242025-11-13T08:09:04.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187252025-11-13T08:09:04.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187262025-11-13T08:09:04.656ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
187272025-11-13T08:09:04.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187282025-11-13T08:09:04.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187292025-11-13T08:09:04.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187302025-11-13T08:09:04.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187312025-11-13T08:09:04.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187322025-11-13T08:09:04.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187332025-11-13T08:09:04.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187342025-11-13T08:09:04.664ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
187352025-11-13T08:09:04.682ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
187362025-11-13T08:09:04.682ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = d00f4add-2b1d-4d87-8ba8-788f041ad77f file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
187372025-11-13T08:09:04.682ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
187382025-11-13T08:09:04.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
187392025-11-13T08:09:04.697ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187402025-11-13T08:09:04.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187412025-11-13T08:09:04.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187422025-11-13T08:09:04.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187432025-11-13T08:09:04.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187442025-11-13T08:09:04.702ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
187452025-11-13T08:09:04.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187462025-11-13T08:09:04.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187472025-11-13T08:09:04.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187482025-11-13T08:09:04.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187492025-11-13T08:09:04.707ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187502025-11-13T08:09:04.710ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
187512025-11-13T08:09:04.710ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187522025-11-13T08:09:04.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187532025-11-13T08:09:04.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187542025-11-13T08:09:04.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187552025-11-13T08:09:04.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187562025-11-13T08:09:04.716ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
187572025-11-13T08:09:04.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187582025-11-13T08:09:04.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187592025-11-13T08:09:04.719ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187602025-11-13T08:09:04.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187612025-11-13T08:09:04.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187622025-11-13T08:09:04.724ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
187632025-11-13T08:09:04.724ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187642025-11-13T08:09:04.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187652025-11-13T08:09:04.726ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187662025-11-13T08:09:04.726ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187672025-11-13T08:09:04.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187682025-11-13T08:09:04.730ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
187692025-11-13T08:09:04.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187702025-11-13T08:09:04.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187712025-11-13T08:09:04.766ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
187722025-11-13T08:09:04.766ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
187732025-11-13T08:09:04.766ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 1fada824-4738-49a3-b211-2d1f6bc9d067 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
187742025-11-13T08:09:04.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187752025-11-13T08:09:04.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187762025-11-13T08:09:04.767ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
187772025-11-13T08:09:04.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
187782025-11-13T08:09:04.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
187792025-11-13T08:09:04.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
187802025-11-13T08:09:04.773ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
187812025-11-13T08:09:04.773ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
187822025-11-13T08:09:04.773ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 349.580162ms iteration = 26 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
187832025-11-13T08:09:04.798ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(b936028d-8e78-4156-af5a-22c410c90fd2, 0), (7bad19a7-d3d3-4161-9b30-7a7b23a70a0e, 0), (3cbe92e3-7f28-4466-b3c2-56fa669e6e95, 0)]
187842025-11-13T08:09:04.798ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6553, tv_nsec: 151048988 } })
187852025-11-13T08:09:04.831ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6553, tv_nsec: 183959545 } })
187862025-11-13T08:09:04.832ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
187872025-11-13T08:09:04.832ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
187882025-11-13T08:09:04.832ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
187892025-11-13T08:09:04.832ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "95739f00-ea83-4d80-b3d2-0c7e776c74a5", "content-length": "476", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
187902025-11-13T08:09:04.832ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
187912025-11-13T08:09:04.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "903f7f4e-0b14-451b-984c-8ebc83d2cbd4", "content-length": "734", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
187922025-11-13T08:09:04.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
187932025-11-13T08:09:04.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35994c7f-9f78-4434-a5a7-2fabc4673a83", "content-length": "111", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
187942025-11-13T08:09:04.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
187952025-11-13T08:09:04.833ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4275ba80-cc7b-4414-b09b-04b3acf75d01", "content-length": "111", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
187962025-11-13T08:09:04.833ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
187972025-11-13T08:09:04.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "965ff941-e25d-4ce1-8f53-c28309ed9e69", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
187982025-11-13T08:09:04.834ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
187992025-11-13T08:09:04.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "499edb0f-e67d-40fc-982b-e4facc824df5", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188002025-11-13T08:09:04.834ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
188012025-11-13T08:09:04.834ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6237806b-542e-4d81-8c68-96e8e64e8a84", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188022025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
188032025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "869e041b-e687-4c87-97d6-00ffbf4db588", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188042025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
188052025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "162ccbf5-88b2-4076-890f-e0bb08c2de49", "content-length": "702", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188062025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
188072025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "225c3657-52f1-4f70-b323-4f7cf1191b44", "content-length": "727", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188082025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
188092025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0a93a44-a8b5-4257-8982-55c392d603fd", "content-length": "729", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188102025-11-13T08:09:04.835ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
188112025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5f455ce-3f1c-4c0e-8e0f-901e4532ea78", "content-length": "728", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188122025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
188132025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "916a6d74-9f7f-44cb-8972-97c0195e2a07", "content-length": "722", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188142025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
188152025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "9842214b-e1af-499f-81fc-a19dd0e06b1f", "content-length": "10", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188162025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
188172025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0dbed14-4517-454e-a118-5a492a08e118", "content-length": "144", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188182025-11-13T08:09:04.836ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
188192025-11-13T08:09:04.837ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "979e9634-1bcd-4d1b-b933-3527f0de8b90", "content-length": "147", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188202025-11-13T08:09:04.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
188212025-11-13T08:09:04.837ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "56824b11-39b6-41da-8f7d-038f4244bdb5", "content-length": "109", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188222025-11-13T08:09:04.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
188232025-11-13T08:09:04.837ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7ae9b2a-3cfb-4ee6-b878-077f5f87d0c5", "content-length": "109", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188242025-11-13T08:09:04.837ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
188252025-11-13T08:09:04.838ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b030c666-bb54-434a-ab4c-d239236811d4", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188262025-11-13T08:09:04.838ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
188272025-11-13T08:09:04.838ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e0e877c-48f3-4a5d-93e1-63f1826da692", "content-length": "163", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188282025-11-13T08:09:04.838ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
188292025-11-13T08:09:04.838ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a8e4438-1636-404b-bff7-dfeb9633e0d5", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188302025-11-13T08:09:04.838ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
188312025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d558bf62-6bb1-4812-812e-cabe14003652", "content-length": "166", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188322025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
188332025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f54efe00-3d73-471f-9277-09408fddef61", "content-length": "702", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188342025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
188352025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2f0701f-3774-480a-b1fc-bf85ed11c117", "content-length": "727", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188362025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
188372025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b53809d-0f44-43e0-a071-b2d9b4989cf5", "content-length": "729", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188382025-11-13T08:09:04.839ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
188392025-11-13T08:09:04.840ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86f975fb-a1f5-4705-9d92-3447901b42b2", "content-length": "728", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188402025-11-13T08:09:04.840ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
188412025-11-13T08:09:04.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
188422025-11-13T08:09:04.842ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188432025-11-13T08:09:04.844ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
188442025-11-13T08:09:04.844ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
188452025-11-13T08:09:04.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
188462025-11-13T08:09:04.845ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
188472025-11-13T08:09:04.847ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
188482025-11-13T08:09:04.847ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
188492025-11-13T08:09:04.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
188502025-11-13T08:09:04.885ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
188512025-11-13T08:09:04.885ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
188522025-11-13T08:09:04.970ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3c732be-c494-424b-a0cc-d77ee5f3c583", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188532025-11-13T08:09:04.972ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
188542025-11-13T08:09:04.972ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
188552025-11-13T08:09:04.972ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
188562025-11-13T08:09:04.972ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
188572025-11-13T08:09:04.972ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
188582025-11-13T08:09:04.972ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
188592025-11-13T08:09:04.972ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
188602025-11-13T08:09:04.972ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
188612025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2ee7c6e-94dd-4471-b55e-9203cf308492", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188622025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdcdf92f-b984-4ea9-8af4-f148f086fa2f", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188632025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "55219e29-98eb-48b6-996f-62e6b589f99b", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188642025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
188652025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "28e70524-6c7c-4ad6-befb-5efdb7a18c5c", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188662025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
188672025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
188682025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "af8c71d4-0e69-4abf-bd45-c5dde66a8dfb", "content-length": "64", "date": "Thu, 13 Nov 2025 08:09:04 GMT"} })
188692025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
188702025-11-13T08:09:04.973ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
188712025-11-13T08:09:05.014ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "fad78c17-bb3c-4d70-a273-e03049078c5e", "content-length": "427617", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188722025-11-13T08:09:05.017ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f13a419c-31a1-432d-b76d-0c4fd515ed6f", "content-length": "405697", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188732025-11-13T08:09:05.017ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "2851bd94-41c6-48e7-b121-d423710c1088", "content-length": "406785", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188742025-11-13T08:09:05.017ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee4ac7bb-fc6f-4233-a9c6-7cf8cd58c602", "content-length": "405594", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188752025-11-13T08:09:05.022ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "50eee326-ca74-4e6b-b621-efcc0dd502fa", "content-length": "412732", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188762025-11-13T08:09:05.024ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
188772025-11-13T08:09:05.024ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
188782025-11-13T08:09:05.024ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
188792025-11-13T08:09:05.024ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
188802025-11-13T08:09:05.024ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
188812025-11-13T08:09:05.054ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
188822025-11-13T08:09:05.054ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
188832025-11-13T08:09:05.058ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c5fc7a3-d176-4658-b88c-eea917480b06", "content-length": "124", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188842025-11-13T08:09:05.059ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
188852025-11-13T08:09:05.149ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
188862025-11-13T08:09:05.149ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
188872025-11-13T08:09:05.150ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "5995cbc7-1459-4774-b8e3-480564d5c6fb", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188882025-11-13T08:09:05.150ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
188892025-11-13T08:09:05.152ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
188902025-11-13T08:09:05.152ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
188912025-11-13T08:09:05.152ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "e98c8917-8ed8-4d0b-af77-3840f7de7cf7", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188922025-11-13T08:09:05.152ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
188932025-11-13T08:09:05.153ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
188942025-11-13T08:09:05.153ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
188952025-11-13T08:09:05.154ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6fb9ca2-bbf5-47b0-9c38-0a4e89f36a9a", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:09:05 GMT"} })
188962025-11-13T08:09:05.154ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
188972025-11-13T08:09:05.155ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
188982025-11-13T08:09:05.155ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
188992025-11-13T08:09:05.157ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
189002025-11-13T08:09:05.157ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
189012025-11-13T08:09:05.159ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189022025-11-13T08:09:05.161ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
189032025-11-13T08:09:05.161ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
189042025-11-13T08:09:05.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
189052025-11-13T08:09:05.403ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = f9644618-9762-4080-98b5-efa02367a4b9 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
189062025-11-13T08:09:05.403ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = f9644618-9762-4080-98b5-efa02367a4b9 time_started = 2025-11-13 08:09:04.832210 UTC
189072025-11-13T08:09:05.403ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 54 reason = Dependency
189082025-11-13T08:09:05.403ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
189092025-11-13T08:09:05.403ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.048740208s iteration = 13 status = {"collection_id":"f9644618-9762-4080-98b5-efa02367a4b9","time_done":"2025-11-13 08:09:05.155594 UTC","time_started":"2025-11-13 08:09:04.832210 UTC"}
189102025-11-13T08:09:05.403ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
189112025-11-13T08:09:05.404ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189122025-11-13T08:09:05.408ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
189132025-11-13T08:09:05.408ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
189142025-11-13T08:09:05.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
189152025-11-13T08:09:05.472ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
189162025-11-13T08:09:05.472ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
189172025-11-13T08:09:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 278 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 428e238a-152c-4ebd-a098-5fa1bd8f0160 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
189182025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 116.919299ms iteration = 54 status = {"Loaded":{"collection_id":"5c9c60a5-45dd-45ca-8f27-717b1b56d226","time_loaded":"2025-11-13T08:09:05.403145035Z","time_started":"2025-11-13T08:09:04.881741Z"}}
189192025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 35 reason = Dependency
189202025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 17 reason = Dependency
189212025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
189222025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 27 reason = Dependency
189232025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189242025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189252025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
189262025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189272025-11-13T08:09:05.520ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189282025-11-13T08:09:05.521ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189292025-11-13T08:09:05.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189302025-11-13T08:09:05.522ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189312025-11-13T08:09:05.524ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
189322025-11-13T08:09:05.524ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
189332025-11-13T08:09:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
189342025-11-13T08:09:05.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
189352025-11-13T08:09:05.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189362025-11-13T08:09:05.525ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
189372025-11-13T08:09:05.525ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189382025-11-13T08:09:05.525ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189392025-11-13T08:09:05.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189402025-11-13T08:09:05.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189412025-11-13T08:09:05.527ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189422025-11-13T08:09:05.527ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
189432025-11-13T08:09:05.527ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
189442025-11-13T08:09:05.528ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189452025-11-13T08:09:05.529ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
189462025-11-13T08:09:05.529ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189472025-11-13T08:09:05.529ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189482025-11-13T08:09:05.530ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
189492025-11-13T08:09:05.530ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189502025-11-13T08:09:05.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189512025-11-13T08:09:05.531ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
189522025-11-13T08:09:05.531ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
189532025-11-13T08:09:05.531ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
189542025-11-13T08:09:05.532ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189552025-11-13T08:09:05.532ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189562025-11-13T08:09:05.533ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189572025-11-13T08:09:05.536ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
189582025-11-13T08:09:05.536ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189592025-11-13T08:09:05.536ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189602025-11-13T08:09:05.537ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189612025-11-13T08:09:05.539ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
189622025-11-13T08:09:05.539ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189632025-11-13T08:09:05.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189642025-11-13T08:09:05.545ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
189652025-11-13T08:09:05.545ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189662025-11-13T08:09:05.545ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189672025-11-13T08:09:05.547ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189682025-11-13T08:09:05.549ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
189692025-11-13T08:09:05.549ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189702025-11-13T08:09:05.549ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189712025-11-13T08:09:05.582ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 62.236664ms iteration = 35 status = {"physical_disks_added":0}
189722025-11-13T08:09:05.593ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189732025-11-13T08:09:05.595ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
189742025-11-13T08:09:05.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
189752025-11-13T08:09:05.596ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
189762025-11-13T08:09:05.597ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
189772025-11-13T08:09:05.597ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 77.79162ms iteration = 17 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"5c9c60a5-45dd-45ca-8f27-717b1b56d226","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
189782025-11-13T08:09:05.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189792025-11-13T08:09:05.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189802025-11-13T08:09:05.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189812025-11-13T08:09:05.610ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
189822025-11-13T08:09:05.610ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189832025-11-13T08:09:05.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189842025-11-13T08:09:05.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189852025-11-13T08:09:05.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
189862025-11-13T08:09:05.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189872025-11-13T08:09:05.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189882025-11-13T08:09:05.617ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189892025-11-13T08:09:05.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189902025-11-13T08:09:05.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189912025-11-13T08:09:05.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189922025-11-13T08:09:05.621ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189932025-11-13T08:09:05.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
189942025-11-13T08:09:05.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189952025-11-13T08:09:05.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189962025-11-13T08:09:05.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
189972025-11-13T08:09:05.625ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
189982025-11-13T08:09:05.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
189992025-11-13T08:09:05.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
190002025-11-13T08:09:05.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190012025-11-13T08:09:05.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190022025-11-13T08:09:05.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190032025-11-13T08:09:05.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
190042025-11-13T08:09:05.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190052025-11-13T08:09:05.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190062025-11-13T08:09:05.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190072025-11-13T08:09:05.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190082025-11-13T08:09:05.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190092025-11-13T08:09:05.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
190102025-11-13T08:09:05.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190112025-11-13T08:09:05.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190122025-11-13T08:09:05.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190132025-11-13T08:09:05.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190142025-11-13T08:09:05.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190152025-11-13T08:09:05.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
190162025-11-13T08:09:05.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190172025-11-13T08:09:05.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190182025-11-13T08:09:05.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190192025-11-13T08:09:05.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190202025-11-13T08:09:05.650ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190212025-11-13T08:09:05.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
190222025-11-13T08:09:05.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190232025-11-13T08:09:05.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190242025-11-13T08:09:05.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190252025-11-13T08:09:05.666ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190262025-11-13T08:09:05.667ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190272025-11-13T08:09:05.667ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190282025-11-13T08:09:05.668ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190292025-11-13T08:09:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
190302025-11-13T08:09:05.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190312025-11-13T08:09:05.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190322025-11-13T08:09:05.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190332025-11-13T08:09:05.675ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
190342025-11-13T08:09:05.675ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190352025-11-13T08:09:05.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190362025-11-13T08:09:05.678ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190372025-11-13T08:09:05.678ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190382025-11-13T08:09:05.680ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190392025-11-13T08:09:05.680ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190402025-11-13T08:09:05.681ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190412025-11-13T08:09:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
190422025-11-13T08:09:05.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190432025-11-13T08:09:05.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190442025-11-13T08:09:05.685ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190452025-11-13T08:09:05.688ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
190462025-11-13T08:09:05.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190472025-11-13T08:09:05.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190482025-11-13T08:09:05.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190492025-11-13T08:09:05.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190502025-11-13T08:09:05.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190512025-11-13T08:09:05.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190522025-11-13T08:09:05.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190532025-11-13T08:09:05.697ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
190542025-11-13T08:09:05.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190552025-11-13T08:09:05.698ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190562025-11-13T08:09:05.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190572025-11-13T08:09:05.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190582025-11-13T08:09:05.700ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190592025-11-13T08:09:05.704ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
190602025-11-13T08:09:05.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190612025-11-13T08:09:05.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190622025-11-13T08:09:05.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190632025-11-13T08:09:05.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190642025-11-13T08:09:05.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190652025-11-13T08:09:05.761ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
190662025-11-13T08:09:05.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190672025-11-13T08:09:05.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190682025-11-13T08:09:05.764ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190692025-11-13T08:09:05.764ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190702025-11-13T08:09:05.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190712025-11-13T08:09:05.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
190722025-11-13T08:09:05.770ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190732025-11-13T08:09:05.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190742025-11-13T08:09:05.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190752025-11-13T08:09:05.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190762025-11-13T08:09:05.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190772025-11-13T08:09:05.780ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
190782025-11-13T08:09:05.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190792025-11-13T08:09:05.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190802025-11-13T08:09:05.783ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190812025-11-13T08:09:05.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190822025-11-13T08:09:05.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190832025-11-13T08:09:05.786ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
190842025-11-13T08:09:05.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190852025-11-13T08:09:05.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190862025-11-13T08:09:05.823ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
190872025-11-13T08:09:05.823ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
190882025-11-13T08:09:05.824ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = f70d126a-1e88-45fd-920c-38041ad5aed0 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
190892025-11-13T08:09:05.824ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190902025-11-13T08:09:05.824ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190912025-11-13T08:09:05.825ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
190922025-11-13T08:09:05.829ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
190932025-11-13T08:09:05.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
190942025-11-13T08:09:05.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
190952025-11-13T08:09:05.831ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
190962025-11-13T08:09:05.831ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
190972025-11-13T08:09:05.831ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 311.530444ms iteration = 27 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
190982025-11-13T08:09:07.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 122 reason = Timeout
190992025-11-13T08:09:07.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191002025-11-13T08:09:07.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191012025-11-13T08:09:07.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191022025-11-13T08:09:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
191032025-11-13T08:09:07.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191042025-11-13T08:09:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191052025-11-13T08:09:07.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
191062025-11-13T08:09:07.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.784783ms iteration = 122 status = {"config_updated":false}
191072025-11-13T08:09:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 62 reason = Timeout
191082025-11-13T08:09:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 123 reason = Timeout
191092025-11-13T08:09:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
191102025-11-13T08:09:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191112025-11-13T08:09:12.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
191122025-11-13T08:09:12.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191132025-11-13T08:09:12.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191142025-11-13T08:09:12.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191152025-11-13T08:09:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
191162025-11-13T08:09:12.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
191172025-11-13T08:09:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
191182025-11-13T08:09:12.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191192025-11-13T08:09:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
191202025-11-13T08:09:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191212025-11-13T08:09:12.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
191222025-11-13T08:09:12.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
191232025-11-13T08:09:12.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
191242025-11-13T08:09:12.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.18539ms iteration = 123 status = {"config_updated":false}
191252025-11-13T08:09:12.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191262025-11-13T08:09:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
191272025-11-13T08:09:12.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
191282025-11-13T08:09:12.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
191292025-11-13T08:09:12.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 62 reason = Timeout
191302025-11-13T08:09:12.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
191312025-11-13T08:09:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
191322025-11-13T08:09:12.658ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
191332025-11-13T08:09:12.658ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 32.158322ms iteration = 62 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
191342025-11-13T08:09:12.686ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
191352025-11-13T08:09:12.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
191362025-11-13T08:09:12.723ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 81.318528ms iteration = 62 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
191372025-11-13T08:09:15.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 2c443b4a-d474-4171-88fa-80f406507664 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
191382025-11-13T08:09:15.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 2c443b4a-d474-4171-88fa-80f406507664 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
191392025-11-13T08:09:15.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 2c443b4a-d474-4171-88fa-80f406507664 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
191402025-11-13T08:09:15.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3313 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 2c443b4a-d474-4171-88fa-80f406507664 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
191412025-11-13T08:09:15.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9c9e1d4-95a5-4294-87fa-d5bdf318877b uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
191422025-11-13T08:09:15.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9c9e1d4-95a5-4294-87fa-d5bdf318877b roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
191432025-11-13T08:09:15.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9c9e1d4-95a5-4294-87fa-d5bdf318877b resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
191442025-11-13T08:09:15.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2645 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = c9c9e1d4-95a5-4294-87fa-d5bdf318877b response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
191452025-11-13T08:09:15.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 253 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = b6f2f23f-1de8-4b6d-9451-a698c4d1eac2 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
191462025-11-13T08:09:17.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 55 reason = Timeout
191472025-11-13T08:09:17.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 124 reason = Timeout
191482025-11-13T08:09:17.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
191492025-11-13T08:09:17.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191502025-11-13T08:09:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191512025-11-13T08:09:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
191522025-11-13T08:09:17.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191532025-11-13T08:09:17.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191542025-11-13T08:09:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
191552025-11-13T08:09:17.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191562025-11-13T08:09:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
191572025-11-13T08:09:17.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
191582025-11-13T08:09:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191592025-11-13T08:09:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
191602025-11-13T08:09:17.633ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5c9c60a5-45dd-45ca-8f27-717b1b56d226 original_time_started = 2025-11-13 08:09:04.881741 UTC
191612025-11-13T08:09:17.633ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.735702ms iteration = 55 status = {"Loaded":{"collection_id":"5c9c60a5-45dd-45ca-8f27-717b1b56d226","time_loaded":"2025-11-13T08:09:17.626153619Z","time_started":"2025-11-13T08:09:04.881741Z"}}
191622025-11-13T08:09:17.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
191632025-11-13T08:09:17.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.805153ms iteration = 124 status = {"config_updated":false}
191642025-11-13T08:09:17.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 42 reason = Timeout
191652025-11-13T08:09:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 42 reason = Timeout
191662025-11-13T08:09:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
191672025-11-13T08:09:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
191682025-11-13T08:09:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191692025-11-13T08:09:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
191702025-11-13T08:09:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
191712025-11-13T08:09:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191722025-11-13T08:09:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191732025-11-13T08:09:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191742025-11-13T08:09:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
191752025-11-13T08:09:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
191762025-11-13T08:09:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
191772025-11-13T08:09:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
191782025-11-13T08:09:17.653ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.884436ms iteration = 42 status = "NoSitrep"
191792025-11-13T08:09:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
191802025-11-13T08:09:17.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191812025-11-13T08:09:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
191822025-11-13T08:09:17.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
191832025-11-13T08:09:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191842025-11-13T08:09:17.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
191852025-11-13T08:09:17.657ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
191862025-11-13T08:09:17.657ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 9.941166ms iteration = 42 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
191872025-11-13T08:09:22.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 63 reason = Timeout
191882025-11-13T08:09:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 125 reason = Timeout
191892025-11-13T08:09:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
191902025-11-13T08:09:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
191912025-11-13T08:09:22.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
191922025-11-13T08:09:22.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
191932025-11-13T08:09:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191942025-11-13T08:09:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
191952025-11-13T08:09:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
191962025-11-13T08:09:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
191972025-11-13T08:09:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
191982025-11-13T08:09:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
191992025-11-13T08:09:22.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
192002025-11-13T08:09:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
192012025-11-13T08:09:22.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192022025-11-13T08:09:22.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
192032025-11-13T08:09:22.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
192042025-11-13T08:09:22.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.890495ms iteration = 125 status = {"config_updated":false}
192052025-11-13T08:09:22.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192062025-11-13T08:09:22.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
192072025-11-13T08:09:22.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192082025-11-13T08:09:22.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
192092025-11-13T08:09:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 63 reason = Timeout
192102025-11-13T08:09:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
192112025-11-13T08:09:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
192122025-11-13T08:09:22.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
192132025-11-13T08:09:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
192142025-11-13T08:09:22.650ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
192152025-11-13T08:09:22.650ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.747477ms iteration = 63 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
192162025-11-13T08:09:22.678ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 35.482958ms iteration = 63 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
192172025-11-13T08:09:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 307 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 781664fa-9dd3-4a95-8f3f-453de2178d49 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
192182025-11-13T08:09:27.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 126 reason = Timeout
192192025-11-13T08:09:27.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
192202025-11-13T08:09:27.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
192212025-11-13T08:09:27.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192222025-11-13T08:09:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
192232025-11-13T08:09:27.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
192242025-11-13T08:09:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
192252025-11-13T08:09:27.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
192262025-11-13T08:09:27.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.162571ms iteration = 126 status = {"config_updated":false}
192272025-11-13T08:09:30.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 4b095607-54cf-487a-8e12-6bf157b19d04 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
192282025-11-13T08:09:30.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 4b095607-54cf-487a-8e12-6bf157b19d04 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
192292025-11-13T08:09:30.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 4b095607-54cf-487a-8e12-6bf157b19d04 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
192302025-11-13T08:09:30.475ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2693 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = 4b095607-54cf-487a-8e12-6bf157b19d04 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
192312025-11-13T08:09:30.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = aa1bda92-344c-4738-bcc9-a8b8fd7a4846 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
192322025-11-13T08:09:30.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = aa1bda92-344c-4738-bcc9-a8b8fd7a4846 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
192332025-11-13T08:09:30.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = aa1bda92-344c-4738-bcc9-a8b8fd7a4846 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
192342025-11-13T08:09:30.478ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2789 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:37654 req_id = aa1bda92-344c-4738-bcc9-a8b8fd7a4846 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
192352025-11-13T08:09:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 22 reason = Timeout
192362025-11-13T08:09:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 22 reason = Timeout
192372025-11-13T08:09:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
192382025-11-13T08:09:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 22 reason = Timeout
192392025-11-13T08:09:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
192402025-11-13T08:09:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
192412025-11-13T08:09:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
192422025-11-13T08:09:32.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6580, tv_nsec: 963692503 } })
192432025-11-13T08:09:32.616ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6580, tv_nsec: 968415676 } })
192442025-11-13T08:09:32.616ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
192452025-11-13T08:09:32.617ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 10.406834ms iteration = 22 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
192462025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 36 reason = Timeout
192472025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 56 reason = Timeout
192482025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192492025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 33 reason = Timeout
192502025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 64 reason = Timeout
192512025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
192522025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
192532025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 127 reason = Timeout
192542025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192552025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
192562025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192572025-11-13T08:09:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
192582025-11-13T08:09:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
192592025-11-13T08:09:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
192602025-11-13T08:09:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
192612025-11-13T08:09:32.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192622025-11-13T08:09:32.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192632025-11-13T08:09:32.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192642025-11-13T08:09:32.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192652025-11-13T08:09:32.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
192662025-11-13T08:09:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
192672025-11-13T08:09:32.634ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
192682025-11-13T08:09:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
192692025-11-13T08:09:32.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192702025-11-13T08:09:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
192712025-11-13T08:09:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192722025-11-13T08:09:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
192732025-11-13T08:09:32.635ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
192742025-11-13T08:09:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
192752025-11-13T08:09:32.635ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
192762025-11-13T08:09:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
192772025-11-13T08:09:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
192782025-11-13T08:09:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
192792025-11-13T08:09:32.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
192802025-11-13T08:09:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
192812025-11-13T08:09:32.639ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
192822025-11-13T08:09:32.639ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 13.015525ms iteration = 127 status = {"config_updated":false}
192832025-11-13T08:09:32.639ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
192842025-11-13T08:09:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
192852025-11-13T08:09:32.641ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
192862025-11-13T08:09:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
192872025-11-13T08:09:32.641ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 22 reason = Timeout
192882025-11-13T08:09:32.641ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 33 reason = Timeout
192892025-11-13T08:09:32.641ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
192902025-11-13T08:09:32.641ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
192912025-11-13T08:09:32.641ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
192922025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 22 reason = Timeout
192932025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
192942025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 22 reason = Timeout
192952025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
192962025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 22 reason = Timeout
192972025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
192982025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 64 reason = Timeout
192992025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
193002025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
193012025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 22 reason = Timeout
193022025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
193032025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
193042025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 22 reason = Timeout
193052025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
193062025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
193072025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
193082025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
193092025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
193102025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
193112025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
193122025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
193132025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
193142025-11-13T08:09:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
193152025-11-13T08:09:32.645ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193162025-11-13T08:09:32.645ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193172025-11-13T08:09:32.646ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193182025-11-13T08:09:32.646ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5c9c60a5-45dd-45ca-8f27-717b1b56d226 original_time_started = 2025-11-13 08:09:04.881741 UTC
193192025-11-13T08:09:32.646ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 20.265758ms iteration = 56 status = {"Loaded":{"collection_id":"5c9c60a5-45dd-45ca-8f27-717b1b56d226","time_loaded":"2025-11-13T08:09:32.626496603Z","time_started":"2025-11-13T08:09:04.881741Z"}}
193202025-11-13T08:09:32.647ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193212025-11-13T08:09:32.647ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
193222025-11-13T08:09:32.647ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
193232025-11-13T08:09:32.647ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
193242025-11-13T08:09:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
193252025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
193262025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 22 reason = Timeout
193272025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
193282025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 22 reason = Timeout
193292025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
193302025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 43 reason = Timeout
193312025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
193322025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 43 reason = Timeout
193332025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193342025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
193352025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 22 reason = Timeout
193362025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 22 reason = Timeout
193372025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
193382025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
193392025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 22 reason = Timeout
193402025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 22 reason = Timeout
193412025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
193422025-11-13T08:09:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
193432025-11-13T08:09:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
193442025-11-13T08:09:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
193452025-11-13T08:09:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
193462025-11-13T08:09:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
193472025-11-13T08:09:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
193482025-11-13T08:09:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
193492025-11-13T08:09:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6581, tv_nsec: 1541197 } })
193502025-11-13T08:09:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
193512025-11-13T08:09:32.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
193522025-11-13T08:09:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
193532025-11-13T08:09:32.650ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
193542025-11-13T08:09:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
193552025-11-13T08:09:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
193562025-11-13T08:09:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
193572025-11-13T08:09:32.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
193582025-11-13T08:09:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
193592025-11-13T08:09:32.651ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
193602025-11-13T08:09:32.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193612025-11-13T08:09:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
193622025-11-13T08:09:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
193632025-11-13T08:09:32.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
193642025-11-13T08:09:32.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
193652025-11-13T08:09:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
193662025-11-13T08:09:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
193672025-11-13T08:09:32.653ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
193682025-11-13T08:09:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
193692025-11-13T08:09:32.655ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6581, tv_nsec: 7732149 } })
193702025-11-13T08:09:32.655ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
193712025-11-13T08:09:32.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
193722025-11-13T08:09:32.656ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
193732025-11-13T08:09:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
193742025-11-13T08:09:32.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 10.016417ms iteration = 43 status = "NoSitrep"
193752025-11-13T08:09:32.723ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193762025-11-13T08:09:32.726ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
193772025-11-13T08:09:32.726ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
193782025-11-13T08:09:32.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
193792025-11-13T08:09:32.730ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
193802025-11-13T08:09:32.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
193812025-11-13T08:09:32.731ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
193822025-11-13T08:09:32.731ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
193832025-11-13T08:09:32.732ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
193842025-11-13T08:09:32.732ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
193852025-11-13T08:09:32.732ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
193862025-11-13T08:09:32.732ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
193872025-11-13T08:09:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
193882025-11-13T08:09:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
193892025-11-13T08:09:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
193902025-11-13T08:09:32.733ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 84.581813ms iteration = 22 status = {"errors":[],"requests_created_ok":[]}
193912025-11-13T08:09:32.733ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 126.704391ms iteration = 22 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
193922025-11-13T08:09:32.734ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193932025-11-13T08:09:32.735ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
193942025-11-13T08:09:32.735ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
193952025-11-13T08:09:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
193962025-11-13T08:09:32.737ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 88.947609ms iteration = 22 status = {"errors":[],"finish_invoked_ok":[]}
193972025-11-13T08:09:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
193982025-11-13T08:09:32.738ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
193992025-11-13T08:09:32.738ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 90.04202ms iteration = 43 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
194002025-11-13T08:09:32.740ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
194012025-11-13T08:09:32.740ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
194022025-11-13T08:09:32.740ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 91.908486ms iteration = 22 status = {"errors":[],"garbage_collect_requested":[]}
194032025-11-13T08:09:32.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
194042025-11-13T08:09:32.740ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
194052025-11-13T08:09:32.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
194062025-11-13T08:09:32.742ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
194072025-11-13T08:09:32.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
194082025-11-13T08:09:32.748ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194092025-11-13T08:09:32.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194102025-11-13T08:09:32.754ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
194112025-11-13T08:09:32.754ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
194122025-11-13T08:09:32.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
194132025-11-13T08:09:32.756ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
194142025-11-13T08:09:32.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
194152025-11-13T08:09:32.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
194162025-11-13T08:09:32.760ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
194172025-11-13T08:09:32.760ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 118.679593ms iteration = 22 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
194182025-11-13T08:09:32.760ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 119.009719ms iteration = 22 status = 0
194192025-11-13T08:09:32.762ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
194202025-11-13T08:09:32.762ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 136.453092ms iteration = 33 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
194212025-11-13T08:09:32.777ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
194222025-11-13T08:09:32.777ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
194232025-11-13T08:09:32.777ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd6e22b8-ad77-4ac7-afdd-7954cb16ab9e", "content-length": "26", "date": "Thu, 13 Nov 2025 08:09:32 GMT"} })
194242025-11-13T08:09:32.777ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
194252025-11-13T08:09:32.780ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
194262025-11-13T08:09:32.791ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 149.235704ms iteration = 64 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
194272025-11-13T08:09:32.792ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
194282025-11-13T08:09:32.792ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 166.382261ms iteration = 64 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
194292025-11-13T08:09:32.794ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 145.572611ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
194302025-11-13T08:09:32.795ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
194312025-11-13T08:09:32.795ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
194322025-11-13T08:09:32.796ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b49e3a55-d085-4b75-81d8-7b66f4b79c6c", "content-length": "26", "date": "Thu, 13 Nov 2025 08:09:32 GMT"} })
194332025-11-13T08:09:32.796ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
194342025-11-13T08:09:32.798ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
194352025-11-13T08:09:32.798ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
194362025-11-13T08:09:32.798ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
194372025-11-13T08:09:32.798ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 156.350734ms iteration = 22 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
194382025-11-13T08:09:32.802ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 160.672209ms iteration = 22 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
194392025-11-13T08:09:32.814ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
194402025-11-13T08:09:32.814ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f1de881-ad12-4600-88bc-32d06adeef61", "content-length": "78", "date": "Thu, 13 Nov 2025 08:09:32 GMT"} })
194412025-11-13T08:09:32.814ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
194422025-11-13T08:09:32.814ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
194432025-11-13T08:09:32.814ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
194442025-11-13T08:09:32.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
194452025-11-13T08:09:32.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
194462025-11-13T08:09:32.815ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
194472025-11-13T08:09:32.815ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 166.729148ms iteration = 22 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
194482025-11-13T08:09:32.816ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
194492025-11-13T08:09:32.817ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
194502025-11-13T08:09:32.817ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
194512025-11-13T08:09:32.820ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
194522025-11-13T08:09:32.820ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
194532025-11-13T08:09:32.821ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
194542025-11-13T08:09:32.821ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
194552025-11-13T08:09:32.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
194562025-11-13T08:09:32.821ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
194572025-11-13T08:09:32.822ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
194582025-11-13T08:09:32.822ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
194592025-11-13T08:09:32.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
194602025-11-13T08:09:32.823ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
194612025-11-13T08:09:32.823ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
194622025-11-13T08:09:32.823ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "f639a5bc-c4a7-469c-a079-da3d80a5039a", "content-length": "81", "date": "Thu, 13 Nov 2025 08:09:32 GMT"} })
194632025-11-13T08:09:32.846ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
194642025-11-13T08:09:32.846ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
194652025-11-13T08:09:32.857ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 230.684506ms iteration = 36 status = {"physical_disks_added":0}
194662025-11-13T08:09:32.857ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
194672025-11-13T08:09:32.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
194682025-11-13T08:09:32.860ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194692025-11-13T08:09:32.864ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
194702025-11-13T08:09:32.864ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
194712025-11-13T08:09:32.864ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
194722025-11-13T08:09:32.866ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
194732025-11-13T08:09:32.869ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
194742025-11-13T08:09:32.869ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
194752025-11-13T08:09:32.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
194762025-11-13T08:09:32.872ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
194772025-11-13T08:09:32.872ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
194782025-11-13T08:09:32.872ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b6bd6a4-6f76-4198-82a4-5593cb9476d4", "content-length": "81", "date": "Thu, 13 Nov 2025 08:09:32 GMT"} })
194792025-11-13T08:09:32.872ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
194802025-11-13T08:09:32.872ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
194812025-11-13T08:09:32.873ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "60d380c8-cc24-4252-a618-cf9419d67e88", "content-length": "81", "date": "Thu, 13 Nov 2025 08:09:32 GMT"} })
194822025-11-13T08:09:32.874ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 226.271849ms iteration = 22 status = {"disabled":false,"errors":[],"sps":[]}
194832025-11-13T08:09:32.914ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6581, tv_nsec: 266461945 } })
194842025-11-13T08:09:32.978ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 371.910162ms iteration = 22 status = {}
194852025-11-13T08:09:33.007ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
194862025-11-13T08:09:33.007ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
194872025-11-13T08:09:33.012ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a32140e6-7f39-4bd4-939b-28caafd7776c", "content-length": "372", "date": "Thu, 13 Nov 2025 08:09:33 GMT"} })
194882025-11-13T08:09:33.012ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
194892025-11-13T08:09:33.013ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
194902025-11-13T08:09:33.030ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
194912025-11-13T08:09:33.030ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "271f1367-a975-47eb-8cbf-b8e002e04c08", "content-length": "691", "date": "Thu, 13 Nov 2025 08:09:33 GMT"} })
194922025-11-13T08:09:33.030ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
194932025-11-13T08:09:33.030ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
194942025-11-13T08:09:33.030ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
194952025-11-13T08:09:33.032ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 390.637329ms iteration = 22 status = {}
194962025-11-13T08:09:33.061ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
194972025-11-13T08:09:33.062ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
194982025-11-13T08:09:33.065ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
194992025-11-13T08:09:33.065ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195002025-11-13T08:09:33.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
195012025-11-13T08:09:33.067ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
195022025-11-13T08:09:33.067ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
195032025-11-13T08:09:33.069ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "07b77cd8-3992-4cb6-91f4-2c2bae32b471", "date": "Thu, 13 Nov 2025 08:09:33 GMT"} })
195042025-11-13T08:09:33.069ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
195052025-11-13T08:09:33.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
195062025-11-13T08:09:33.072ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
195072025-11-13T08:09:33.073ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
195082025-11-13T08:09:33.075ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
195092025-11-13T08:09:33.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
195102025-11-13T08:09:33.078ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
195112025-11-13T08:09:33.078ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
195122025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
195132025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
195142025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
195152025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
195162025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
195172025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
195182025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
195192025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
195202025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
195212025-11-13T08:09:33.080ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
195222025-11-13T08:09:33.081ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
195232025-11-13T08:09:33.081ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195242025-11-13T08:09:33.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
195252025-11-13T08:09:33.084ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
195262025-11-13T08:09:33.084ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
195272025-11-13T08:09:33.084ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "91e95d71-2840-400f-b77a-ba2260e73dce", "date": "Thu, 13 Nov 2025 08:09:33 GMT"} })
195282025-11-13T08:09:33.086ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 444.108771ms iteration = 33 status = {}
195292025-11-13T08:09:33.273ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "45707311-280b-496a-91e1-1d80bf4ca6bb", "date": "Thu, 13 Nov 2025 08:09:33 GMT"} })
195302025-11-13T08:09:33.273ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
195312025-11-13T08:09:33.273ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
195322025-11-13T08:09:33.273ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a860518b-19a6-4a5e-a859-1a378a6f6267", "content-length": "682", "date": "Thu, 13 Nov 2025 08:09:33 GMT"} })
195332025-11-13T08:09:33.273ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195342025-11-13T08:09:33.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
195352025-11-13T08:09:33.277ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195362025-11-13T08:09:33.277ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
195372025-11-13T08:09:33.281ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195382025-11-13T08:09:33.281ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
195392025-11-13T08:09:33.283ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
195402025-11-13T08:09:33.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
195412025-11-13T08:09:33.291ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 648.920728ms iteration = 22 status = {}
195422025-11-13T08:09:35.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 315 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 82f6f965-97b6-4186-8572-0300a9b862a8 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
195432025-11-13T08:09:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 128 reason = Timeout
195442025-11-13T08:09:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195452025-11-13T08:09:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195462025-11-13T08:09:37.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195472025-11-13T08:09:37.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
195482025-11-13T08:09:37.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195492025-11-13T08:09:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195502025-11-13T08:09:37.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
195512025-11-13T08:09:37.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.506068ms iteration = 128 status = {"config_updated":false}
195522025-11-13T08:09:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 65 reason = Timeout
195532025-11-13T08:09:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 129 reason = Timeout
195542025-11-13T08:09:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195552025-11-13T08:09:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195562025-11-13T08:09:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195572025-11-13T08:09:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195582025-11-13T08:09:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195592025-11-13T08:09:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195602025-11-13T08:09:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
195612025-11-13T08:09:42.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195622025-11-13T08:09:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195632025-11-13T08:09:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
195642025-11-13T08:09:42.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195652025-11-13T08:09:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195662025-11-13T08:09:42.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195672025-11-13T08:09:42.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
195682025-11-13T08:09:42.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.710701ms iteration = 129 status = {"config_updated":false}
195692025-11-13T08:09:42.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195702025-11-13T08:09:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195712025-11-13T08:09:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
195722025-11-13T08:09:42.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
195732025-11-13T08:09:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
195742025-11-13T08:09:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 65 reason = Timeout
195752025-11-13T08:09:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
195762025-11-13T08:09:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
195772025-11-13T08:09:42.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
195782025-11-13T08:09:42.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
195792025-11-13T08:09:42.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.784365ms iteration = 65 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
195802025-11-13T08:09:42.654ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
195812025-11-13T08:09:42.654ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.504391ms iteration = 65 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
195822025-11-13T08:09:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 313 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 01c08485-64d4-419a-9ea2-974adbde526e response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
195832025-11-13T08:09:47.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 57 reason = Timeout
195842025-11-13T08:09:47.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 130 reason = Timeout
195852025-11-13T08:09:47.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
195862025-11-13T08:09:47.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195872025-11-13T08:09:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
195882025-11-13T08:09:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195892025-11-13T08:09:47.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195902025-11-13T08:09:47.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
195912025-11-13T08:09:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
195922025-11-13T08:09:47.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
195932025-11-13T08:09:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
195942025-11-13T08:09:47.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
195952025-11-13T08:09:47.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
195962025-11-13T08:09:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
195972025-11-13T08:09:47.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
195982025-11-13T08:09:47.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.187101ms iteration = 130 status = {"config_updated":false}
195992025-11-13T08:09:47.633ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5c9c60a5-45dd-45ca-8f27-717b1b56d226 original_time_started = 2025-11-13 08:09:04.881741 UTC
196002025-11-13T08:09:47.633ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.761063ms iteration = 57 status = {"Loaded":{"collection_id":"5c9c60a5-45dd-45ca-8f27-717b1b56d226","time_loaded":"2025-11-13T08:09:47.626064653Z","time_started":"2025-11-13T08:09:04.881741Z"}}
196012025-11-13T08:09:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 44 reason = Timeout
196022025-11-13T08:09:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
196032025-11-13T08:09:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 44 reason = Timeout
196042025-11-13T08:09:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196052025-11-13T08:09:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
196062025-11-13T08:09:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196072025-11-13T08:09:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196082025-11-13T08:09:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196092025-11-13T08:09:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196102025-11-13T08:09:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196112025-11-13T08:09:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
196122025-11-13T08:09:47.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
196132025-11-13T08:09:47.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
196142025-11-13T08:09:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
196152025-11-13T08:09:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.968318ms iteration = 44 status = "NoSitrep"
196162025-11-13T08:09:47.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196172025-11-13T08:09:47.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196182025-11-13T08:09:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
196192025-11-13T08:09:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
196202025-11-13T08:09:47.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196212025-11-13T08:09:47.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
196222025-11-13T08:09:47.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
196232025-11-13T08:09:47.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.419175ms iteration = 44 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
196242025-11-13T08:09:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 66 reason = Timeout
196252025-11-13T08:09:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 131 reason = Timeout
196262025-11-13T08:09:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196272025-11-13T08:09:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
196282025-11-13T08:09:52.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196292025-11-13T08:09:52.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
196302025-11-13T08:09:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196312025-11-13T08:09:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196322025-11-13T08:09:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
196332025-11-13T08:09:52.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196342025-11-13T08:09:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196352025-11-13T08:09:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
196362025-11-13T08:09:52.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
196372025-11-13T08:09:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
196382025-11-13T08:09:52.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196392025-11-13T08:09:52.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196402025-11-13T08:09:52.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
196412025-11-13T08:09:52.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.749862ms iteration = 131 status = {"config_updated":false}
196422025-11-13T08:09:52.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196432025-11-13T08:09:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
196442025-11-13T08:09:52.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
196452025-11-13T08:09:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
196462025-11-13T08:09:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 66 reason = Timeout
196472025-11-13T08:09:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
196482025-11-13T08:09:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
196492025-11-13T08:09:52.651ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
196502025-11-13T08:09:52.651ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.600044ms iteration = 66 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
196512025-11-13T08:09:52.677ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
196522025-11-13T08:09:52.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
196532025-11-13T08:09:52.709ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 66.899005ms iteration = 66 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
196542025-11-13T08:09:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 323 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 6ec9a306-a1e9-4636-abf9-a57c37a5badf response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
196552025-11-13T08:09:57.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 132 reason = Timeout
196562025-11-13T08:09:57.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
196572025-11-13T08:09:57.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
196582025-11-13T08:09:57.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
196592025-11-13T08:09:57.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
196602025-11-13T08:09:57.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
196612025-11-13T08:09:57.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
196622025-11-13T08:09:57.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
196632025-11-13T08:09:57.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.797853ms iteration = 132 status = {"config_updated":false}
196642025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 12 reason = Timeout
196652025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 12 reason = Timeout
196662025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
196672025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
196682025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 12 reason = Timeout
196692025-11-13T08:10:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 08:00:02.607559719 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
196702025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 12 reason = Timeout
196712025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
196722025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
196732025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
196742025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 12 reason = Timeout
196752025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
196762025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 23 reason = Timeout
196772025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
196782025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 23 reason = Timeout
196792025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
196802025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 14 reason = Timeout
196812025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 23 reason = Timeout
196822025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 14 reason = Timeout
196832025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 12 reason = Timeout
196842025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 13 reason = Timeout
196852025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
196862025-11-13T08:10:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
196872025-11-13T08:10:02.607ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = ExternalNetworking step id = Ensure
196882025-11-13T08:10:02.607ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
196892025-11-13T08:10:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
196902025-11-13T08:10:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
196912025-11-13T08:10:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
196922025-11-13T08:10:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
196932025-11-13T08:10:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
196942025-11-13T08:10:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
196952025-11-13T08:10:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
196962025-11-13T08:10:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6610, tv_nsec: 960814473 } })
196972025-11-13T08:10:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6610, tv_nsec: 960849813 } })
196982025-11-13T08:10:02.609ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
196992025-11-13T08:10:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.53827ms iteration = 12 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
197002025-11-13T08:10:02.609ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
197012025-11-13T08:10:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.413638ms iteration = 12 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
197022025-11-13T08:10:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
197032025-11-13T08:10:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
197042025-11-13T08:10:02.610ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197052025-11-13T08:10:02.610ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
197062025-11-13T08:10:02.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
197072025-11-13T08:10:02.611ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 08:00:02.607559719 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
197082025-11-13T08:10:02.611ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.269774ms iteration = 12 status = {"expiration":"2025-11-13T08:00:02.607559719Z","pruned":[]}
197092025-11-13T08:10:02.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6610, tv_nsec: 964158409 } })
197102025-11-13T08:10:02.612ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6610, tv_nsec: 964924494 } })
197112025-11-13T08:10:02.613ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
197122025-11-13T08:10:02.615ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
197132025-11-13T08:10:02.615ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
197142025-11-13T08:10:02.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
197152025-11-13T08:10:02.618ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 12 reason = Timeout
197162025-11-13T08:10:02.618ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
197172025-11-13T08:10:02.618ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
197182025-11-13T08:10:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
197192025-11-13T08:10:02.619ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 28 reason = Timeout
197202025-11-13T08:10:02.619ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197212025-11-13T08:10:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197222025-11-13T08:10:02.623ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
197232025-11-13T08:10:02.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
197242025-11-13T08:10:02.624ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 16.79641ms iteration = 23 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
197252025-11-13T08:10:02.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
197262025-11-13T08:10:02.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
197272025-11-13T08:10:02.628ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 34 reason = Timeout
197282025-11-13T08:10:02.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
197292025-11-13T08:10:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
197302025-11-13T08:10:02.629ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 37 reason = Timeout
197312025-11-13T08:10:02.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
197322025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
197332025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 67 reason = Timeout
197342025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197352025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
197362025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 133 reason = Timeout
197372025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
197382025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 58 reason = Timeout
197392025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
197402025-11-13T08:10:02.630ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
197412025-11-13T08:10:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
197422025-11-13T08:10:02.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
197432025-11-13T08:10:02.638ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
197442025-11-13T08:10:02.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
197452025-11-13T08:10:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
197462025-11-13T08:10:02.639ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197472025-11-13T08:10:02.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197482025-11-13T08:10:02.640ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197492025-11-13T08:10:02.641ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197502025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 12 reason = Timeout
197512025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
197522025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 23 reason = Timeout
197532025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 34 reason = Timeout
197542025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
197552025-11-13T08:10:02.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
197562025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 23 reason = Timeout
197572025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
197582025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
197592025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 67 reason = Timeout
197602025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
197612025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 23 reason = Timeout
197622025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
197632025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 12 reason = Timeout
197642025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 23 reason = Timeout
197652025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
197662025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
197672025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 23 reason = Timeout
197682025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
197692025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 23 reason = Timeout
197702025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
197712025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
197722025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197732025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
197742025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
197752025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
197762025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
197772025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
197782025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
197792025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
197802025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
197812025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
197822025-11-13T08:10:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
197832025-11-13T08:10:02.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
197842025-11-13T08:10:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
197852025-11-13T08:10:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 12 reason = Timeout
197862025-11-13T08:10:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
197872025-11-13T08:10:02.645ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197882025-11-13T08:10:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
197892025-11-13T08:10:02.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
197902025-11-13T08:10:02.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
197912025-11-13T08:10:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
197922025-11-13T08:10:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
197932025-11-13T08:10:02.647ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
197942025-11-13T08:10:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
197952025-11-13T08:10:02.647ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
197962025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 12 reason = Timeout
197972025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
197982025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
197992025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 23 reason = Timeout
198002025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 23 reason = Timeout
198012025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
198022025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
198032025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 23 reason = Timeout
198042025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 23 reason = Timeout
198052025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
198062025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
198072025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
198082025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 45 reason = Timeout
198092025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
198102025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 23 reason = Timeout
198112025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 23 reason = Timeout
198122025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
198132025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
198142025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 45 reason = Timeout
198152025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
198162025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
198172025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
198182025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
198192025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
198202025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
198212025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198222025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 12 reason = Timeout
198232025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
198242025-11-13T08:10:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
198252025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
198262025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
198272025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
198282025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
198292025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
198302025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
198312025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 3.719144ms iteration = 12 status = {"errors":[],"found_port_ok":[]}
198322025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
198332025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
198342025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 12 reason = Timeout
198352025-11-13T08:10:02.649ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
198362025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
198372025-11-13T08:10:02.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6611, tv_nsec: 1793969 } })
198382025-11-13T08:10:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
198392025-11-13T08:10:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
198402025-11-13T08:10:02.650ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
198412025-11-13T08:10:02.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
198422025-11-13T08:10:02.651ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
198432025-11-13T08:10:02.651ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
198442025-11-13T08:10:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
198452025-11-13T08:10:02.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
198462025-11-13T08:10:02.652ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
198472025-11-13T08:10:02.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
198482025-11-13T08:10:02.652ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198492025-11-13T08:10:02.653ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
198502025-11-13T08:10:02.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
198512025-11-13T08:10:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
198522025-11-13T08:10:02.655ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
198532025-11-13T08:10:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
198542025-11-13T08:10:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
198552025-11-13T08:10:02.657ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
198562025-11-13T08:10:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
198572025-11-13T08:10:02.659ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
198582025-11-13T08:10:02.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
198592025-11-13T08:10:02.663ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 15.607467ms iteration = 23 status = {"errors":[],"requests_created_ok":[]}
198602025-11-13T08:10:02.674ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
198612025-11-13T08:10:02.674ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 26.855658ms iteration = 12 status = {"by_rx":{},"error":null}
198622025-11-13T08:10:02.676ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
198632025-11-13T08:10:02.676ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
198642025-11-13T08:10:02.700ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198652025-11-13T08:10:02.706ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
198662025-11-13T08:10:02.708ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198672025-11-13T08:10:02.709ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
198682025-11-13T08:10:02.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
198692025-11-13T08:10:02.713ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
198702025-11-13T08:10:02.717ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
198712025-11-13T08:10:02.717ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
198722025-11-13T08:10:02.717ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
198732025-11-13T08:10:02.717ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
198742025-11-13T08:10:02.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
198752025-11-13T08:10:02.721ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198762025-11-13T08:10:02.725ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 5c9c60a5-45dd-45ca-8f27-717b1b56d226 original_time_started = 2025-11-13 08:09:04.881741 UTC
198772025-11-13T08:10:02.725ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 96.020397ms iteration = 58 status = {"Loaded":{"collection_id":"5c9c60a5-45dd-45ca-8f27-717b1b56d226","time_loaded":"2025-11-13T08:10:02.629958620Z","time_started":"2025-11-13T08:09:04.881741Z"}}
198782025-11-13T08:10:02.726ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198792025-11-13T08:10:02.727ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
198802025-11-13T08:10:02.727ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
198812025-11-13T08:10:02.727ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
198822025-11-13T08:10:02.728ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
198832025-11-13T08:10:02.728ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
198842025-11-13T08:10:02.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
198852025-11-13T08:10:02.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
198862025-11-13T08:10:02.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198872025-11-13T08:10:02.728ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198882025-11-13T08:10:02.729ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
198892025-11-13T08:10:02.729ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 99.565067ms iteration = 133 status = {"config_updated":false}
198902025-11-13T08:10:02.730ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
198912025-11-13T08:10:02.730ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
198922025-11-13T08:10:02.730ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 88.624822ms iteration = 12 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
198932025-11-13T08:10:02.731ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
198942025-11-13T08:10:02.731ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
198952025-11-13T08:10:02.731ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
198962025-11-13T08:10:02.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
198972025-11-13T08:10:02.732ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
198982025-11-13T08:10:02.733ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
198992025-11-13T08:10:02.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
199002025-11-13T08:10:02.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
199012025-11-13T08:10:02.733ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6611, tv_nsec: 85451313 } })
199022025-11-13T08:10:02.733ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 91.096281ms iteration = 23 status = 0
199032025-11-13T08:10:02.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
199042025-11-13T08:10:02.733ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
199052025-11-13T08:10:02.733ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 91.760354ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
199062025-11-13T08:10:02.734ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
199072025-11-13T08:10:02.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199082025-11-13T08:10:02.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
199092025-11-13T08:10:02.734ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
199102025-11-13T08:10:02.734ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
199112025-11-13T08:10:02.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
199122025-11-13T08:10:02.735ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
199132025-11-13T08:10:02.735ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
199142025-11-13T08:10:02.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
199152025-11-13T08:10:02.736ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199162025-11-13T08:10:02.737ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
199172025-11-13T08:10:02.737ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199182025-11-13T08:10:02.738ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
199192025-11-13T08:10:02.738ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 90.03507ms iteration = 45 status = "NoSitrep"
199202025-11-13T08:10:02.739ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 90.876856ms iteration = 12 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
199212025-11-13T08:10:02.740ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 98.1817ms iteration = 67 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
199222025-11-13T08:10:02.740ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199232025-11-13T08:10:02.740ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
199242025-11-13T08:10:02.740ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 98.523206ms iteration = 23 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
199252025-11-13T08:10:02.740ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
199262025-11-13T08:10:02.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
199272025-11-13T08:10:02.740ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
199282025-11-13T08:10:02.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
199292025-11-13T08:10:02.741ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199302025-11-13T08:10:02.743ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
199312025-11-13T08:10:02.743ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
199322025-11-13T08:10:02.743ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
199332025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
199342025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
199352025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
199362025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
199372025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
199382025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
199392025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
199402025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
199412025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
199422025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
199432025-11-13T08:10:02.749ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
199442025-11-13T08:10:02.749ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 119.220234ms iteration = 34 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
199452025-11-13T08:10:02.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199462025-11-13T08:10:02.751ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199472025-11-13T08:10:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
199482025-11-13T08:10:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
199492025-11-13T08:10:02.761ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199502025-11-13T08:10:02.764ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
199512025-11-13T08:10:02.764ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
199522025-11-13T08:10:02.764ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 145.596262ms iteration = 12 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
199532025-11-13T08:10:02.764ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 115.99485ms iteration = 23 status = {"errors":[],"garbage_collect_requested":[]}
199542025-11-13T08:10:02.764ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
199552025-11-13T08:10:02.764ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 116.077962ms iteration = 45 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
199562025-11-13T08:10:02.765ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
199572025-11-13T08:10:02.765ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
199582025-11-13T08:10:02.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
199592025-11-13T08:10:02.766ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
199602025-11-13T08:10:02.767ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 119.49144ms iteration = 23 status = {"errors":[],"finish_invoked_ok":[]}
199612025-11-13T08:10:02.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
199622025-11-13T08:10:02.770ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
199632025-11-13T08:10:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
199642025-11-13T08:10:02.772ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
199652025-11-13T08:10:02.772ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199662025-11-13T08:10:02.775ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
199672025-11-13T08:10:02.776ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
199682025-11-13T08:10:02.776ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
199692025-11-13T08:10:02.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
199702025-11-13T08:10:02.786ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
199712025-11-13T08:10:02.786ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
199722025-11-13T08:10:02.787ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
199732025-11-13T08:10:02.787ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
199742025-11-13T08:10:02.787ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
199752025-11-13T08:10:02.787ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
199762025-11-13T08:10:02.787ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
199772025-11-13T08:10:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
199782025-11-13T08:10:02.790ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
199792025-11-13T08:10:02.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
199802025-11-13T08:10:02.792ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
199812025-11-13T08:10:02.794ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
199822025-11-13T08:10:02.794ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 164.917823ms iteration = 67 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
199832025-11-13T08:10:02.795ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
199842025-11-13T08:10:02.797ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
199852025-11-13T08:10:02.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
199862025-11-13T08:10:02.798ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
199872025-11-13T08:10:02.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
199882025-11-13T08:10:02.799ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
199892025-11-13T08:10:02.801ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
199902025-11-13T08:10:02.803ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
199912025-11-13T08:10:02.804ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = SledList step id = Fetch
199922025-11-13T08:10:02.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
199932025-11-13T08:10:02.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
199942025-11-13T08:10:02.806ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
199952025-11-13T08:10:02.806ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
199962025-11-13T08:10:02.806ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
199972025-11-13T08:10:02.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
199982025-11-13T08:10:02.811ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
199992025-11-13T08:10:02.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
200002025-11-13T08:10:02.814ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
200012025-11-13T08:10:02.814ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
200022025-11-13T08:10:02.814ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
200032025-11-13T08:10:02.814ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 166.566755ms iteration = 23 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
200042025-11-13T08:10:02.814ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
200052025-11-13T08:10:02.816ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200062025-11-13T08:10:02.819ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
200072025-11-13T08:10:02.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
200082025-11-13T08:10:02.821ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200092025-11-13T08:10:02.822ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
200102025-11-13T08:10:02.823ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
200112025-11-13T08:10:02.823ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
200122025-11-13T08:10:02.825ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
200132025-11-13T08:10:02.825ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
200142025-11-13T08:10:02.826ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
200152025-11-13T08:10:02.828ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
200162025-11-13T08:10:02.828ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
200172025-11-13T08:10:02.828ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
200182025-11-13T08:10:02.828ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 221.358452ms iteration = 12 status = {"generation":1}
200192025-11-13T08:10:02.838ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
200202025-11-13T08:10:02.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
200212025-11-13T08:10:02.840ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200222025-11-13T08:10:02.842ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
200232025-11-13T08:10:02.842ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
200242025-11-13T08:10:02.843ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c90d1b3-d040-4630-be7b-7cddddbca1f5", "content-length": "26", "date": "Thu, 13 Nov 2025 08:10:02 GMT"} })
200252025-11-13T08:10:02.843ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
200262025-11-13T08:10:02.844ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
200272025-11-13T08:10:02.844ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
200282025-11-13T08:10:02.845ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
200292025-11-13T08:10:02.848ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
200302025-11-13T08:10:02.849ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
200312025-11-13T08:10:02.849ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200322025-11-13T08:10:02.853ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
200332025-11-13T08:10:02.853ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
200342025-11-13T08:10:02.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
200352025-11-13T08:10:02.864ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
200362025-11-13T08:10:02.865ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
200372025-11-13T08:10:02.878ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
200382025-11-13T08:10:02.878ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
200392025-11-13T08:10:02.878ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
200402025-11-13T08:10:02.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200412025-11-13T08:10:02.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
200422025-11-13T08:10:02.879ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
200432025-11-13T08:10:02.879ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
200442025-11-13T08:10:02.879ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 237.51905ms iteration = 23 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
200452025-11-13T08:10:02.881ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200462025-11-13T08:10:02.882ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
200472025-11-13T08:10:02.882ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200482025-11-13T08:10:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200492025-11-13T08:10:02.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200502025-11-13T08:10:02.887ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
200512025-11-13T08:10:02.887ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
200522025-11-13T08:10:02.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
200532025-11-13T08:10:02.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
200542025-11-13T08:10:02.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
200552025-11-13T08:10:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
200562025-11-13T08:10:02.892ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
200572025-11-13T08:10:02.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
200582025-11-13T08:10:02.894ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
200592025-11-13T08:10:02.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
200602025-11-13T08:10:02.908ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 260.743837ms iteration = 23 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
200612025-11-13T08:10:02.909ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
200622025-11-13T08:10:02.909ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
200632025-11-13T08:10:02.909ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
200642025-11-13T08:10:02.909ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 301.509797ms iteration = 12 status = {"generation":2}
200652025-11-13T08:10:02.923ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(3)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
200662025-11-13T08:10:02.923ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 315.984852ms iteration = 12 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
200672025-11-13T08:10:02.925ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
200682025-11-13T08:10:02.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
200692025-11-13T08:10:02.928ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
200702025-11-13T08:10:02.933ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 325.55016ms iteration = 23 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
200712025-11-13T08:10:02.938ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
200722025-11-13T08:10:02.961ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
200732025-11-13T08:10:02.961ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "0402fbae-401b-4eba-9231-3ba1160af61a", "content-length": "78", "date": "Thu, 13 Nov 2025 08:10:02 GMT"} })
200742025-11-13T08:10:02.961ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
200752025-11-13T08:10:02.961ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
200762025-11-13T08:10:02.962ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
200772025-11-13T08:10:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
200782025-11-13T08:10:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
200792025-11-13T08:10:02.962ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
200802025-11-13T08:10:02.965ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
200812025-11-13T08:10:02.965ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
200822025-11-13T08:10:02.970ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
200832025-11-13T08:10:02.970ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
200842025-11-13T08:10:02.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
200852025-11-13T08:10:02.970ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
200862025-11-13T08:10:02.970ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
200872025-11-13T08:10:02.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
200882025-11-13T08:10:02.971ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
200892025-11-13T08:10:02.971ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
200902025-11-13T08:10:02.972ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ce68517-f222-438d-bd20-0fa0580535d5", "content-length": "26", "date": "Thu, 13 Nov 2025 08:10:02 GMT"} })
200912025-11-13T08:10:02.972ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
200922025-11-13T08:10:02.974ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
200932025-11-13T08:10:02.974ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 332.036608ms iteration = 12 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
200942025-11-13T08:10:02.974ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
200952025-11-13T08:10:02.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
200962025-11-13T08:10:02.988ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
200972025-11-13T08:10:02.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
200982025-11-13T08:10:02.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
200992025-11-13T08:10:02.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201002025-11-13T08:10:02.994ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
201012025-11-13T08:10:02.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201022025-11-13T08:10:02.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201032025-11-13T08:10:02.996ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201042025-11-13T08:10:02.998ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
201052025-11-13T08:10:02.998ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
201062025-11-13T08:10:02.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
201072025-11-13T08:10:02.999ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 370.131127ms iteration = 37 status = {"physical_disks_added":0}
201082025-11-13T08:10:03.020ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
201092025-11-13T08:10:03.020ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
201102025-11-13T08:10:03.020ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "28b9576d-687b-4afc-92d8-ce943b879265", "content-length": "372", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
201112025-11-13T08:10:03.020ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201122025-11-13T08:10:03.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201132025-11-13T08:10:03.032ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
201142025-11-13T08:10:03.037ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
201152025-11-13T08:10:03.037ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
201162025-11-13T08:10:03.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
201172025-11-13T08:10:03.041ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = DeployNexusRecords step id = Ensure
201182025-11-13T08:10:03.041ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201192025-11-13T08:10:03.042ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201202025-11-13T08:10:03.042ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
201212025-11-13T08:10:03.042ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
201222025-11-13T08:10:03.043ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d68d54c-50fa-4166-8c3b-17339b1d321a", "content-length": "81", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
201232025-11-13T08:10:03.043ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
201242025-11-13T08:10:03.043ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
201252025-11-13T08:10:03.043ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "c71d0285-201e-47a9-a99c-8d64d2b0fae6", "content-length": "81", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
201262025-11-13T08:10:03.044ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201272025-11-13T08:10:03.046ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
201282025-11-13T08:10:03.046ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201292025-11-13T08:10:03.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201302025-11-13T08:10:03.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
201312025-11-13T08:10:03.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
201322025-11-13T08:10:03.052ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 444.829966ms iteration = 23 status = {}
201332025-11-13T08:10:03.052ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201342025-11-13T08:10:03.056ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
201352025-11-13T08:10:03.061ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
201362025-11-13T08:10:03.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201372025-11-13T08:10:03.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201382025-11-13T08:10:03.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201392025-11-13T08:10:03.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201402025-11-13T08:10:03.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201412025-11-13T08:10:03.080ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
201422025-11-13T08:10:03.080ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201432025-11-13T08:10:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201442025-11-13T08:10:03.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201452025-11-13T08:10:03.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201462025-11-13T08:10:03.090ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
201472025-11-13T08:10:03.090ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
201482025-11-13T08:10:03.091ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "b84e996b-69b7-4486-97ff-1608f8d053af", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
201492025-11-13T08:10:03.091ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201502025-11-13T08:10:03.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201512025-11-13T08:10:03.094ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201522025-11-13T08:10:03.094ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
201532025-11-13T08:10:03.094ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
201542025-11-13T08:10:03.094ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201552025-11-13T08:10:03.094ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "daecf97e-4b92-4a05-94cd-71347dc5cd89", "content-length": "81", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
201562025-11-13T08:10:03.096ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 448.744002ms iteration = 23 status = {"disabled":false,"errors":[],"sps":[]}
201572025-11-13T08:10:03.099ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
201582025-11-13T08:10:03.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
201592025-11-13T08:10:03.104ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
201602025-11-13T08:10:03.104ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
201612025-11-13T08:10:03.104ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f2122384-639e-4d47-97e5-ad0d81868ec2", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
201622025-11-13T08:10:03.105ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
201632025-11-13T08:10:03.106ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "7e31c700-88aa-410f-ab7d-ef5e31f94238", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
201642025-11-13T08:10:03.106ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
201652025-11-13T08:10:03.106ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = SledAgent step id = Ensure
201662025-11-13T08:10:03.106ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 464.546943ms iteration = 34 status = {}
201672025-11-13T08:10:03.108ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "a52c9172-94a5-4301-afe6-2d604904ef70", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
201682025-11-13T08:10:03.108ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
201692025-11-13T08:10:03.111ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
201702025-11-13T08:10:03.111ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 503.465388ms iteration = 14 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
201712025-11-13T08:10:03.119ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6611, tv_nsec: 472089214 } })
201722025-11-13T08:10:03.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201732025-11-13T08:10:03.136ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
201742025-11-13T08:10:03.136ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201752025-11-13T08:10:03.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201762025-11-13T08:10:03.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201772025-11-13T08:10:03.142ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
201782025-11-13T08:10:03.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201792025-11-13T08:10:03.143ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201802025-11-13T08:10:03.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201812025-11-13T08:10:03.145ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201822025-11-13T08:10:03.147ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201832025-11-13T08:10:03.151ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
201842025-11-13T08:10:03.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201852025-11-13T08:10:03.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201862025-11-13T08:10:03.154ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201872025-11-13T08:10:03.154ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201882025-11-13T08:10:03.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201892025-11-13T08:10:03.160ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
201902025-11-13T08:10:03.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201912025-11-13T08:10:03.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201922025-11-13T08:10:03.162ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201932025-11-13T08:10:03.162ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201942025-11-13T08:10:03.164ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
201952025-11-13T08:10:03.168ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
201962025-11-13T08:10:03.168ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
201972025-11-13T08:10:03.168ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
201982025-11-13T08:10:03.169ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
201992025-11-13T08:10:03.169ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "76404b52-aab2-489b-8b6c-3b8734b33d72", "content-length": "691", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
202002025-11-13T08:10:03.169ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
202012025-11-13T08:10:03.169ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
202022025-11-13T08:10:03.169ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
202032025-11-13T08:10:03.172ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 530.505279ms iteration = 23 status = {}
202042025-11-13T08:10:03.182ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
202052025-11-13T08:10:03.182ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "82af4a5d-dcb2-47a8-96db-344b62e42dbf", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
202062025-11-13T08:10:03.182ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
202072025-11-13T08:10:03.185ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
202082025-11-13T08:10:03.185ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 535.769743ms iteration = 12 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
202092025-11-13T08:10:03.185ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 35 reason = Dependency
202102025-11-13T08:10:03.185ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
202112025-11-13T08:10:03.185ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
202122025-11-13T08:10:03.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
202132025-11-13T08:10:03.186ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202142025-11-13T08:10:03.187ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202152025-11-13T08:10:03.187ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202162025-11-13T08:10:03.190ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
202172025-11-13T08:10:03.190ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
202182025-11-13T08:10:03.190ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202192025-11-13T08:10:03.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
202202025-11-13T08:10:03.190ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202212025-11-13T08:10:03.191ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202222025-11-13T08:10:03.192ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202232025-11-13T08:10:03.195ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
202242025-11-13T08:10:03.195ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
202252025-11-13T08:10:03.195ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
202262025-11-13T08:10:03.196ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
202272025-11-13T08:10:03.196ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202282025-11-13T08:10:03.196ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202292025-11-13T08:10:03.198ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202302025-11-13T08:10:03.202ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
202312025-11-13T08:10:03.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202322025-11-13T08:10:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202332025-11-13T08:10:03.205ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
202342025-11-13T08:10:03.205ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202352025-11-13T08:10:03.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202362025-11-13T08:10:03.206ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "c149b787-63ad-4e23-98bc-23877f4d1a31", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
202372025-11-13T08:10:03.206ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
202382025-11-13T08:10:03.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202392025-11-13T08:10:03.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202402025-11-13T08:10:03.209ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "c8a4a741-7b7b-484f-88cc-4d7589c17082", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
202412025-11-13T08:10:03.209ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
202422025-11-13T08:10:03.209ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202432025-11-13T08:10:03.211ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "961d7106-3709-41ac-823d-bffb8b20915a", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
202442025-11-13T08:10:03.211ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
202452025-11-13T08:10:03.213ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
202462025-11-13T08:10:03.213ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202472025-11-13T08:10:03.213ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202482025-11-13T08:10:03.214ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
202492025-11-13T08:10:03.214ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 606.621276ms iteration = 14 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
202502025-11-13T08:10:03.216ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
202512025-11-13T08:10:03.217ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "2e9168e4-ef2f-45c1-9a85-abfc42e38e7e", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
202522025-11-13T08:10:03.219ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = FirewallRules step id = Ensure
202532025-11-13T08:10:03.219ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202542025-11-13T08:10:03.219ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202552025-11-13T08:10:03.221ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202562025-11-13T08:10:03.221ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202572025-11-13T08:10:03.223ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202582025-11-13T08:10:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202592025-11-13T08:10:03.226ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202602025-11-13T08:10:03.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202612025-11-13T08:10:03.231ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202622025-11-13T08:10:03.231ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202632025-11-13T08:10:03.267ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202642025-11-13T08:10:03.268ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202652025-11-13T08:10:03.269ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202662025-11-13T08:10:03.270ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
202672025-11-13T08:10:03.270ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202682025-11-13T08:10:03.270ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202692025-11-13T08:10:03.271ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
202702025-11-13T08:10:03.271ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202712025-11-13T08:10:03.272ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202722025-11-13T08:10:03.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202732025-11-13T08:10:03.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202742025-11-13T08:10:03.274ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
202752025-11-13T08:10:03.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
202762025-11-13T08:10:03.280ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
202772025-11-13T08:10:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
202782025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
202792025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
202802025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
202812025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
202822025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
202832025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
202842025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
202852025-11-13T08:10:03.284ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
202862025-11-13T08:10:03.285ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
202872025-11-13T08:10:03.285ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
202882025-11-13T08:10:03.286ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
202892025-11-13T08:10:03.286ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
202902025-11-13T08:10:03.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
202912025-11-13T08:10:03.289ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
202922025-11-13T08:10:03.289ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
202932025-11-13T08:10:03.321ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
202942025-11-13T08:10:03.323ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
202952025-11-13T08:10:03.323ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
202962025-11-13T08:10:03.325ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d05ebc8-5ad3-496a-81ec-f4b1a6e73a52", "content-length": "372", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
202972025-11-13T08:10:03.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
202982025-11-13T08:10:03.325ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
202992025-11-13T08:10:03.325ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203002025-11-13T08:10:03.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203012025-11-13T08:10:03.325ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
203022025-11-13T08:10:03.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203032025-11-13T08:10:03.327ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203042025-11-13T08:10:03.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203052025-11-13T08:10:03.330ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
203062025-11-13T08:10:03.377ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
203072025-11-13T08:10:03.377ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
203082025-11-13T08:10:03.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203092025-11-13T08:10:03.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203102025-11-13T08:10:03.381ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "3c15af27-d042-45e8-a1e0-67cb3951af1c", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
203112025-11-13T08:10:03.381ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
203122025-11-13T08:10:03.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
203132025-11-13T08:10:03.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203142025-11-13T08:10:03.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203152025-11-13T08:10:03.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203162025-11-13T08:10:03.383ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203172025-11-13T08:10:03.383ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203182025-11-13T08:10:03.384ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
203192025-11-13T08:10:03.384ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
203202025-11-13T08:10:03.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203212025-11-13T08:10:03.387ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
203222025-11-13T08:10:03.387ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
203232025-11-13T08:10:03.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
203242025-11-13T08:10:03.388ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203252025-11-13T08:10:03.388ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203262025-11-13T08:10:03.388ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
203272025-11-13T08:10:03.390ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203282025-11-13T08:10:03.390ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
203292025-11-13T08:10:03.390ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
203302025-11-13T08:10:03.390ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203312025-11-13T08:10:03.391ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "e77b741d-3dc1-4daf-9f6a-30a386417660", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
203322025-11-13T08:10:03.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203332025-11-13T08:10:03.393ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203342025-11-13T08:10:03.393ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 208.167551ms iteration = 35 status = {}
203352025-11-13T08:10:03.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203362025-11-13T08:10:03.395ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203372025-11-13T08:10:03.396ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203382025-11-13T08:10:03.396ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203392025-11-13T08:10:03.397ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203402025-11-13T08:10:03.398ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203412025-11-13T08:10:03.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203422025-11-13T08:10:03.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203432025-11-13T08:10:03.401ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203442025-11-13T08:10:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
203452025-11-13T08:10:03.405ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203462025-11-13T08:10:03.405ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203472025-11-13T08:10:03.407ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203482025-11-13T08:10:03.407ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203492025-11-13T08:10:03.408ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203502025-11-13T08:10:03.408ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203512025-11-13T08:10:03.411ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203522025-11-13T08:10:03.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203532025-11-13T08:10:03.413ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203542025-11-13T08:10:03.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203552025-11-13T08:10:03.415ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203562025-11-13T08:10:03.415ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203572025-11-13T08:10:03.416ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203582025-11-13T08:10:03.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203592025-11-13T08:10:03.418ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203602025-11-13T08:10:03.422ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203612025-11-13T08:10:03.422ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203622025-11-13T08:10:03.423ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
203632025-11-13T08:10:03.423ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203642025-11-13T08:10:03.423ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203652025-11-13T08:10:03.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203662025-11-13T08:10:03.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203672025-11-13T08:10:03.424ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203682025-11-13T08:10:03.424ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203692025-11-13T08:10:03.425ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203702025-11-13T08:10:03.426ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203712025-11-13T08:10:03.426ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203722025-11-13T08:10:03.427ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203732025-11-13T08:10:03.429ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
203742025-11-13T08:10:03.429ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203752025-11-13T08:10:03.429ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203762025-11-13T08:10:03.431ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203772025-11-13T08:10:03.431ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203782025-11-13T08:10:03.432ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
203792025-11-13T08:10:03.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203802025-11-13T08:10:03.432ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203812025-11-13T08:10:03.432ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203822025-11-13T08:10:03.435ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
203832025-11-13T08:10:03.435ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203842025-11-13T08:10:03.435ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203852025-11-13T08:10:03.436ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203862025-11-13T08:10:03.436ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203872025-11-13T08:10:03.437ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203882025-11-13T08:10:03.437ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203892025-11-13T08:10:03.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203902025-11-13T08:10:03.438ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203912025-11-13T08:10:03.438ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
203922025-11-13T08:10:03.439ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203932025-11-13T08:10:03.440ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203942025-11-13T08:10:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
203952025-11-13T08:10:03.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
203962025-11-13T08:10:03.442ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
203972025-11-13T08:10:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
203982025-11-13T08:10:03.442ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
203992025-11-13T08:10:03.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204002025-11-13T08:10:03.443ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204012025-11-13T08:10:03.443ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
204022025-11-13T08:10:03.444ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
204032025-11-13T08:10:03.444ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204042025-11-13T08:10:03.445ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204052025-11-13T08:10:03.445ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204062025-11-13T08:10:03.446ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204072025-11-13T08:10:03.446ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204082025-11-13T08:10:03.447ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
204092025-11-13T08:10:03.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
204102025-11-13T08:10:03.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
204112025-11-13T08:10:03.448ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204122025-11-13T08:10:03.452ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
204132025-11-13T08:10:03.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204142025-11-13T08:10:03.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204152025-11-13T08:10:03.453ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204162025-11-13T08:10:03.454ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204172025-11-13T08:10:03.455ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204182025-11-13T08:10:03.455ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204192025-11-13T08:10:03.457ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204202025-11-13T08:10:03.462ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
204212025-11-13T08:10:03.462ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204222025-11-13T08:10:03.462ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204232025-11-13T08:10:03.469ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
204242025-11-13T08:10:03.469ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204252025-11-13T08:10:03.470ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204262025-11-13T08:10:03.471ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204272025-11-13T08:10:03.473ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
204282025-11-13T08:10:03.473ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204292025-11-13T08:10:03.473ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204302025-11-13T08:10:03.475ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
204312025-11-13T08:10:03.475ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
204322025-11-13T08:10:03.475ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
204332025-11-13T08:10:03.495ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
204342025-11-13T08:10:03.495ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
204352025-11-13T08:10:03.495ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = bd020310-f08f-4baf-a1f6-f28cd2a54f07 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
204362025-11-13T08:10:03.495ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
204372025-11-13T08:10:03.495ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
204382025-11-13T08:10:03.497ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204392025-11-13T08:10:03.500ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
204402025-11-13T08:10:03.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
204412025-11-13T08:10:03.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
204422025-11-13T08:10:03.503ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
204432025-11-13T08:10:03.503ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
204442025-11-13T08:10:03.503ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 884.460738ms iteration = 28 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
204452025-11-13T08:10:03.516ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "64048e80-8541-46db-89f3-4cdc7ad86371", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
204462025-11-13T08:10:03.516ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
204472025-11-13T08:10:03.516ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
204482025-11-13T08:10:03.521ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d0ab713-e5ca-48fe-88cf-bbc0ad55f846", "content-length": "682", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
204492025-11-13T08:10:03.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
204502025-11-13T08:10:03.522ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
204512025-11-13T08:10:03.525ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
204522025-11-13T08:10:03.526ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
204532025-11-13T08:10:03.529ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
204542025-11-13T08:10:03.530ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
204552025-11-13T08:10:03.533ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
204562025-11-13T08:10:03.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
204572025-11-13T08:10:03.541ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 899.584825ms iteration = 23 status = {}
204582025-11-13T08:10:03.605ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
204592025-11-13T08:10:03.605ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
204602025-11-13T08:10:03.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204612025-11-13T08:10:03.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204622025-11-13T08:10:03.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204632025-11-13T08:10:03.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204642025-11-13T08:10:03.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204652025-11-13T08:10:03.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
204662025-11-13T08:10:03.645ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
204672025-11-13T08:10:03.646ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "9c4bf3c7-b4b1-4a67-bbdf-a2775f4cddd5", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
204682025-11-13T08:10:03.647ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
204692025-11-13T08:10:03.647ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = Dns step id = Ensure
204702025-11-13T08:10:03.647ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
204712025-11-13T08:10:03.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204722025-11-13T08:10:03.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204732025-11-13T08:10:03.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204742025-11-13T08:10:03.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204752025-11-13T08:10:03.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204762025-11-13T08:10:03.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
204772025-11-13T08:10:03.664ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 4
204782025-11-13T08:10:03.664ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
204792025-11-13T08:10:03.687ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
204802025-11-13T08:10:03.687ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
204812025-11-13T08:10:03.687ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
204822025-11-13T08:10:03.687ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204832025-11-13T08:10:03.687ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204842025-11-13T08:10:03.689ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204852025-11-13T08:10:03.692ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
204862025-11-13T08:10:03.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204872025-11-13T08:10:03.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204882025-11-13T08:10:03.694ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
204892025-11-13T08:10:03.694ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:48812
204902025-11-13T08:10:03.694ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:48812 req_id = 29f78514-8227-49e3-9d7c-5557b848ea33 uri = /metrics/producers version = 1.0.0
204912025-11-13T08:10:03.694ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:48812 req_id = 29f78514-8227-49e3-9d7c-5557b848ea33 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
204922025-11-13T08:10:03.694ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:48812 req_id = 29f78514-8227-49e3-9d7c-5557b848ea33 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
204932025-11-13T08:10:03.729ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
204942025-11-13T08:10:03.729ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
204952025-11-13T08:10:03.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
204962025-11-13T08:10:03.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
204972025-11-13T08:10:03.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
204982025-11-13T08:10:03.733ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
204992025-11-13T08:10:03.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205002025-11-13T08:10:03.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205012025-11-13T08:10:03.736ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
205022025-11-13T08:10:03.736ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
205032025-11-13T08:10:03.736ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
205042025-11-13T08:10:03.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205052025-11-13T08:10:03.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205062025-11-13T08:10:03.737ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205072025-11-13T08:10:03.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205082025-11-13T08:10:03.738ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205092025-11-13T08:10:03.741ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
205102025-11-13T08:10:03.743ZINFOnexus: assigned collector to new producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d file = nexus/src/app/oximeter.rs:76 producer_id = bcb58129-79cc-406c-aa16-455a5cd9f031
205112025-11-13T08:10:03.743ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 49261 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:48812 req_id = 29f78514-8227-49e3-9d7c-5557b848ea33 response_code = 201 uri = /metrics/producers
205122025-11-13T08:10:03.743ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "29f78514-8227-49e3-9d7c-5557b848ea33", "content-length": "41", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
205132025-11-13T08:10:03.746ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
205142025-11-13T08:10:03.785ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
205152025-11-13T08:10:03.785ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
205162025-11-13T08:10:03.785ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
205172025-11-13T08:10:03.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205182025-11-13T08:10:03.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205192025-11-13T08:10:03.787ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205202025-11-13T08:10:03.790ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
205212025-11-13T08:10:03.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205222025-11-13T08:10:03.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205232025-11-13T08:10:03.793ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
205242025-11-13T08:10:03.793ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
205252025-11-13T08:10:03.793ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205262025-11-13T08:10:03.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205272025-11-13T08:10:03.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205282025-11-13T08:10:03.798ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
205292025-11-13T08:10:03.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205302025-11-13T08:10:03.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205312025-11-13T08:10:03.800ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
205322025-11-13T08:10:03.800ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
205332025-11-13T08:10:03.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205342025-11-13T08:10:03.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205352025-11-13T08:10:03.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205362025-11-13T08:10:03.804ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
205372025-11-13T08:10:03.804ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205382025-11-13T08:10:03.804ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205392025-11-13T08:10:03.806ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205402025-11-13T08:10:03.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205412025-11-13T08:10:03.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205422025-11-13T08:10:03.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205432025-11-13T08:10:03.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205442025-11-13T08:10:03.811ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
205452025-11-13T08:10:03.813ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
205462025-11-13T08:10:03.813ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
205472025-11-13T08:10:03.813ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = OmicronZones step id = Cleanup
205482025-11-13T08:10:03.813ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = OmicronZones step id = Cleanup
205492025-11-13T08:10:03.813ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = PhysicalDisks step id = Cleanup
205502025-11-13T08:10:03.813ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = Clickhouse step id = Ensure
205512025-11-13T08:10:03.813ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = Clickhouse step id = Ensure
205522025-11-13T08:10:03.844ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
205532025-11-13T08:10:03.849ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "bba1d330-b57e-440b-b757-8b4cf5c3e6e0", "date": "Thu, 13 Nov 2025 08:10:03 GMT"} })
205542025-11-13T08:10:03.850ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = SupportBundles step id = Cleanup
205552025-11-13T08:10:03.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205562025-11-13T08:10:03.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205572025-11-13T08:10:03.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205582025-11-13T08:10:03.853ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
205592025-11-13T08:10:03.853ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205602025-11-13T08:10:03.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205612025-11-13T08:10:03.854ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205622025-11-13T08:10:03.854ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205632025-11-13T08:10:03.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205642025-11-13T08:10:03.858ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
205652025-11-13T08:10:03.903ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = OmicronZones step id = Cleanup
205662025-11-13T08:10:03.903ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
205672025-11-13T08:10:03.903ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
205682025-11-13T08:10:03.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205692025-11-13T08:10:03.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205702025-11-13T08:10:03.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205712025-11-13T08:10:03.906ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
205722025-11-13T08:10:03.906ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205732025-11-13T08:10:03.906ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205742025-11-13T08:10:03.937ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
205752025-11-13T08:10:03.938ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
205762025-11-13T08:10:03.938ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = Cockroach step id = Ensure
205772025-11-13T08:10:03.938ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
205782025-11-13T08:10:03.938ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
205792025-11-13T08:10:03.968ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
205802025-11-13T08:10:03.968ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = 441aaac8-3ba0-4f87-867b-643bc940ef9c step component = MgsUpdates step id = Ensure
205812025-11-13T08:10:03.968ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Dependency
205822025-11-13T08:10:03.968ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 35 reason = Dependency
205832025-11-13T08:10:03.968ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
205842025-11-13T08:10:03.968ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
205852025-11-13T08:10:03.968ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.361069718s iteration = 13 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3240,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":196554814,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":196554765,"secs":0}},"event_index":1,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":196570674,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":237507089,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":237506879,"secs":0}},"event_index":2,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":434093974,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":64324675,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":64324615,"secs":0}},"event_index":3,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":498433349,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":113153085,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":113153035,"secs":0}},"event_index":4,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":611600074,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":428060075,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":428060025,"secs":0}},"event_index":5,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":39673119,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":166088145,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":166088115,"secs":0}},"event_index":6,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":205768224,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4831,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":4791,"secs":0}},"event_index":7,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":205783115,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2440,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2400,"secs":0}},"event_index":8,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":205787785,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":4170,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4130,"secs":0}},"event_index":9,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":205794555,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2510,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2460,"secs":0}},"event_index":10,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":205799045,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36715752,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36715692,"secs":0}},"event_index":11,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":242516907,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":53121174,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":53121134,"secs":0}},"event_index":12,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":295647991,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":34509328,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":34509278,"secs":0}},"event_index":13,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":330172530,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":30550881,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":30550841,"secs":0}},"event_index":14,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":360732681,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4430,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4390,"secs":0}},"event_index":15,"execution_id":"bc2d0707-6c52-4a1d-87c7-1f975e13f373","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":360743511,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
205862025-11-13T08:10:03.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
205872025-11-13T08:10:03.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
205882025-11-13T08:10:03.970ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205892025-11-13T08:10:03.970ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
205902025-11-13T08:10:03.972ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
205912025-11-13T08:10:03.972ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
205922025-11-13T08:10:03.972ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
205932025-11-13T08:10:03.972ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
205942025-11-13T08:10:03.972ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
205952025-11-13T08:10:03.973ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
205962025-11-13T08:10:03.994ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(3cbe92e3-7f28-4466-b3c2-56fa669e6e95, 0), (41569d69-1e42-4dd0-b148-1d962bb1cb70, 0), (f9644618-9762-4080-98b5-efa02367a4b9, 0), (5c9c60a5-45dd-45ca-8f27-717b1b56d226, 0)] collection_id = 3cbe92e3-7f28-4466-b3c2-56fa669e6e95
205972025-11-13T08:10:03.995ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
205982025-11-13T08:10:03.995ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
205992025-11-13T08:10:04.015ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
206002025-11-13T08:10:04.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
206012025-11-13T08:10:04.018ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206022025-11-13T08:10:04.020ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
206032025-11-13T08:10:04.020ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
206042025-11-13T08:10:04.020ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
206052025-11-13T08:10:04.055ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
206062025-11-13T08:10:04.055ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 86.849477ms iteration = 35 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
206072025-11-13T08:10:04.161ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 3cbe92e3-7f28-4466-b3c2-56fa669e6e95 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
206082025-11-13T08:10:04.161ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
206092025-11-13T08:10:04.161ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
206102025-11-13T08:10:04.165ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(41569d69-1e42-4dd0-b148-1d962bb1cb70, 0), (f9644618-9762-4080-98b5-efa02367a4b9, 0), (5c9c60a5-45dd-45ca-8f27-717b1b56d226, 0)]
206112025-11-13T08:10:04.166ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6612, tv_nsec: 518502576 } })
206122025-11-13T08:10:04.197ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6612, tv_nsec: 550242920 } })
206132025-11-13T08:10:04.198ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
206142025-11-13T08:10:04.198ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
206152025-11-13T08:10:04.198ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
206162025-11-13T08:10:04.199ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbf17114-1935-475c-b8f7-81c59887e2b7", "content-length": "476", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206172025-11-13T08:10:04.199ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
206182025-11-13T08:10:04.199ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "80eb6e31-591c-415c-bb82-492fd43631f4", "content-length": "734", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206192025-11-13T08:10:04.199ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
206202025-11-13T08:10:04.199ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "27f18b7f-810c-47c3-ad64-6f7e501f6c8f", "content-length": "111", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206212025-11-13T08:10:04.199ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
206222025-11-13T08:10:04.200ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b309e7e5-f8a7-4ac7-be79-092151c949ec", "content-length": "111", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206232025-11-13T08:10:04.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
206242025-11-13T08:10:04.200ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c252634-5213-4ba1-840b-2444decd8d77", "content-length": "163", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206252025-11-13T08:10:04.200ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
206262025-11-13T08:10:04.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dedb5599-cb93-4a1a-a768-d9e91381b913", "content-length": "163", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206272025-11-13T08:10:04.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
206282025-11-13T08:10:04.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6958d33d-52c1-400a-bb02-ca908f9c3f57", "content-length": "166", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206292025-11-13T08:10:04.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
206302025-11-13T08:10:04.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90ba35d4-97a7-49e6-916a-8912a196fd3e", "content-length": "166", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206312025-11-13T08:10:04.201ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
206322025-11-13T08:10:04.201ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81cfdab0-57f0-4660-b3ba-73c7c9e41ca8", "content-length": "702", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206332025-11-13T08:10:04.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
206342025-11-13T08:10:04.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ee25ab8-1a53-44b7-8cae-8ba434ce6167", "content-length": "727", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206352025-11-13T08:10:04.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
206362025-11-13T08:10:04.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeece20e-109a-4c75-b639-e9f980bc3257", "content-length": "729", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206372025-11-13T08:10:04.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
206382025-11-13T08:10:04.202ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "476d0c91-9667-4241-95c1-8f69a2581993", "content-length": "728", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206392025-11-13T08:10:04.202ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
206402025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "856d1cf4-793a-48e4-a33a-57510b826084", "content-length": "722", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206412025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
206422025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bb5a228-4e35-4bb1-a96f-f756d33c7e48", "content-length": "10", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206432025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
206442025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fb2b106-2150-42f3-9948-847dc5d855fa", "content-length": "144", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206452025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
206462025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fb64baf-41f7-4697-9bf3-e44e13fc5265", "content-length": "147", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206472025-11-13T08:10:04.203ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
206482025-11-13T08:10:04.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e13bba40-bd25-4f91-8ff0-7554d3e57a7b", "content-length": "109", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206492025-11-13T08:10:04.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
206502025-11-13T08:10:04.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "25eb1491-b90f-4185-b048-3c3e3b2c3806", "content-length": "109", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206512025-11-13T08:10:04.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
206522025-11-13T08:10:04.204ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0135e63-2449-4bb7-a6a8-4636518bde87", "content-length": "163", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206532025-11-13T08:10:04.204ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
206542025-11-13T08:10:04.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18ac1b9a-3f88-49c7-b080-68ac3d397fc6", "content-length": "163", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206552025-11-13T08:10:04.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
206562025-11-13T08:10:04.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1c8e66a-0d83-4128-b871-db5c0b6b22e1", "content-length": "166", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206572025-11-13T08:10:04.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
206582025-11-13T08:10:04.205ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9925ca5-e5b2-4660-b096-5e01fe2a0af3", "content-length": "166", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206592025-11-13T08:10:04.205ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
206602025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5ae99b1-929e-4df7-b919-626c04827c1e", "content-length": "702", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206612025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
206622025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b0b5bec-e14c-4176-9534-1cc8e73340cf", "content-length": "727", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206632025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
206642025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fc050ae-9e49-40ff-a4b8-cc6e343e0a05", "content-length": "729", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206652025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
206662025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dea2e18-d7d3-4913-a913-921dd6e50286", "content-length": "728", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206672025-11-13T08:10:04.206ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
206682025-11-13T08:10:04.207ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
206692025-11-13T08:10:04.208ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206702025-11-13T08:10:04.210ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
206712025-11-13T08:10:04.210ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
206722025-11-13T08:10:04.210ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
206732025-11-13T08:10:04.212ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
206742025-11-13T08:10:04.214ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
206752025-11-13T08:10:04.214ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
206762025-11-13T08:10:04.214ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
206772025-11-13T08:10:04.247ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
206782025-11-13T08:10:04.247ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
206792025-11-13T08:10:04.308ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3de2e52-f607-4b48-a3f2-6a332e38c64c", "content-length": "95178", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206802025-11-13T08:10:04.310ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
206812025-11-13T08:10:04.310ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
206822025-11-13T08:10:04.310ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
206832025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
206842025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
206852025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
206862025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
206872025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
206882025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f96eba-3e97-46af-8ebd-2e4054f0e27e", "content-length": "64", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206892025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
206902025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b16ae4d3-36d1-41a7-875c-a7da2d703f34", "content-length": "64", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206912025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
206922025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "eea366c1-57d8-49b4-8b4c-fb1a900230cf", "content-length": "64", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206932025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
206942025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "38c211a2-2bb3-42b9-8cd1-0696c30706a4", "content-length": "64", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206952025-11-13T08:10:04.310ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
206962025-11-13T08:10:04.311ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "92e38df4-36b3-4db5-98d5-bf6d0f7d8836", "content-length": "64", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206972025-11-13T08:10:04.311ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
206982025-11-13T08:10:04.352ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "16d48c11-8115-4ff5-86a6-82fa708fbc46", "content-length": "407591", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
206992025-11-13T08:10:04.353ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "593c0fed-a36e-4e58-aa14-2c09ec9290cb", "content-length": "414462", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207002025-11-13T08:10:04.355ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "f20dab2a-a0eb-423d-a7d2-cc549f3d8c26", "content-length": "407668", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207012025-11-13T08:10:04.357ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd02ae62-34cb-479b-a515-12a1b4bd3135", "content-length": "408689", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207022025-11-13T08:10:04.357ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a287192-a76a-48b3-885a-2676a9302f95", "content-length": "429229", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207032025-11-13T08:10:04.360ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
207042025-11-13T08:10:04.360ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
207052025-11-13T08:10:04.360ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
207062025-11-13T08:10:04.360ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
207072025-11-13T08:10:04.360ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
207082025-11-13T08:10:04.395ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
207092025-11-13T08:10:04.395ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
207102025-11-13T08:10:04.398ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "959cceab-1d14-427a-be82-d3c422f0e82c", "content-length": "124", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207112025-11-13T08:10:04.401ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
207122025-11-13T08:10:04.494ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
207132025-11-13T08:10:04.494ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
207142025-11-13T08:10:04.495ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "082477a4-021f-4cb1-965d-34a67a321ae6", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207152025-11-13T08:10:04.495ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
207162025-11-13T08:10:04.497ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
207172025-11-13T08:10:04.497ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
207182025-11-13T08:10:04.497ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "90eb17fc-5b19-4fa9-b55e-a9906d6e971d", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207192025-11-13T08:10:04.497ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
207202025-11-13T08:10:04.499ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
207212025-11-13T08:10:04.499ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
207222025-11-13T08:10:04.500ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ae83ba8-3cd2-4c96-9698-b77e54dcc927", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:10:04 GMT"} })
207232025-11-13T08:10:04.500ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
207242025-11-13T08:10:04.501ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
207252025-11-13T08:10:04.501ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
207262025-11-13T08:10:04.503ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
207272025-11-13T08:10:04.504ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
207282025-11-13T08:10:04.505ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207292025-11-13T08:10:04.508ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
207302025-11-13T08:10:04.508ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
207312025-11-13T08:10:04.508ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
207322025-11-13T08:10:04.743ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = b8358fe6-a8f3-4e1d-81e4-53d438ed69da file = nexus/db-queries/src/db/datastore/inventory.rs:1613
207332025-11-13T08:10:04.743ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = b8358fe6-a8f3-4e1d-81e4-53d438ed69da time_started = 2025-11-13 08:10:04.198558 UTC
207342025-11-13T08:10:04.743ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 59 reason = Dependency
207352025-11-13T08:10:04.743ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
207362025-11-13T08:10:04.744ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 775.389743ms iteration = 14 status = {"collection_id":"b8358fe6-a8f3-4e1d-81e4-53d438ed69da","time_done":"2025-11-13 08:10:04.501797 UTC","time_started":"2025-11-13 08:10:04.198558 UTC"}
207372025-11-13T08:10:04.744ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
207382025-11-13T08:10:04.745ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207392025-11-13T08:10:04.749ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
207402025-11-13T08:10:04.749ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
207412025-11-13T08:10:04.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
207422025-11-13T08:10:04.802ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
207432025-11-13T08:10:04.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
207442025-11-13T08:10:04.865ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 38 reason = Dependency
207452025-11-13T08:10:04.865ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 18 reason = Dependency
207462025-11-13T08:10:04.865ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 121.897277ms iteration = 59 status = {"Loaded":{"collection_id":"a7967a9a-1e1f-42c0-b592-9f2752cb200d","time_loaded":"2025-11-13T08:10:04.743958380Z","time_started":"2025-11-13T08:10:04.244650Z"}}
207472025-11-13T08:10:04.865ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 29 reason = Dependency
207482025-11-13T08:10:04.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207492025-11-13T08:10:04.865ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
207502025-11-13T08:10:04.865ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
207512025-11-13T08:10:04.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207522025-11-13T08:10:04.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
207532025-11-13T08:10:04.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
207542025-11-13T08:10:04.867ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207552025-11-13T08:10:04.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207562025-11-13T08:10:04.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207572025-11-13T08:10:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
207582025-11-13T08:10:04.870ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
207592025-11-13T08:10:04.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
207602025-11-13T08:10:04.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
207612025-11-13T08:10:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
207622025-11-13T08:10:04.871ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
207632025-11-13T08:10:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207642025-11-13T08:10:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
207652025-11-13T08:10:04.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207662025-11-13T08:10:04.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207672025-11-13T08:10:04.872ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207682025-11-13T08:10:04.873ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
207692025-11-13T08:10:04.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
207702025-11-13T08:10:04.874ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207712025-11-13T08:10:04.875ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
207722025-11-13T08:10:04.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207732025-11-13T08:10:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207742025-11-13T08:10:04.876ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
207752025-11-13T08:10:04.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
207762025-11-13T08:10:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
207772025-11-13T08:10:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
207782025-11-13T08:10:04.877ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
207792025-11-13T08:10:04.877ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
207802025-11-13T08:10:04.878ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207812025-11-13T08:10:04.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207822025-11-13T08:10:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
207832025-11-13T08:10:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
207842025-11-13T08:10:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207852025-11-13T08:10:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207862025-11-13T08:10:04.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
207872025-11-13T08:10:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
207882025-11-13T08:10:04.883ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
207892025-11-13T08:10:04.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
207902025-11-13T08:10:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
207912025-11-13T08:10:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
207922025-11-13T08:10:04.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207932025-11-13T08:10:04.885ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
207942025-11-13T08:10:04.886ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 20.676676ms iteration = 38 status = {"physical_disks_added":0}
207952025-11-13T08:10:04.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
207962025-11-13T08:10:04.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
207972025-11-13T08:10:04.888ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
207982025-11-13T08:10:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
207992025-11-13T08:10:04.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208002025-11-13T08:10:04.888ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208012025-11-13T08:10:04.889ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
208022025-11-13T08:10:04.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
208032025-11-13T08:10:04.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
208042025-11-13T08:10:04.891ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208052025-11-13T08:10:04.893ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
208062025-11-13T08:10:04.893ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
208072025-11-13T08:10:04.894ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
208082025-11-13T08:10:04.895ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208092025-11-13T08:10:04.897ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
208102025-11-13T08:10:04.897ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
208112025-11-13T08:10:04.897ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
208122025-11-13T08:10:04.899ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
208132025-11-13T08:10:04.899ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 33.836125ms iteration = 18 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"a7967a9a-1e1f-42c0-b592-9f2752cb200d","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
208142025-11-13T08:10:04.923ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208152025-11-13T08:10:04.923ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208162025-11-13T08:10:04.925ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208172025-11-13T08:10:04.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208182025-11-13T08:10:04.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208192025-11-13T08:10:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
208202025-11-13T08:10:04.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208212025-11-13T08:10:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208222025-11-13T08:10:04.931ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208232025-11-13T08:10:04.931ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208242025-11-13T08:10:04.932ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208252025-11-13T08:10:04.937ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
208262025-11-13T08:10:04.937ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208272025-11-13T08:10:04.937ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208282025-11-13T08:10:04.939ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208292025-11-13T08:10:04.941ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
208302025-11-13T08:10:04.941ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208312025-11-13T08:10:04.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208322025-11-13T08:10:04.944ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208332025-11-13T08:10:04.944ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208342025-11-13T08:10:04.945ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208352025-11-13T08:10:04.947ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
208362025-11-13T08:10:04.947ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208372025-11-13T08:10:04.948ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208382025-11-13T08:10:04.949ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208392025-11-13T08:10:04.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208402025-11-13T08:10:04.951ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208412025-11-13T08:10:04.953ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
208422025-11-13T08:10:04.953ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208432025-11-13T08:10:04.954ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208442025-11-13T08:10:04.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208452025-11-13T08:10:04.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208462025-11-13T08:10:04.956ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208472025-11-13T08:10:04.959ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
208482025-11-13T08:10:04.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208492025-11-13T08:10:04.959ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208502025-11-13T08:10:04.974ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208512025-11-13T08:10:04.974ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208522025-11-13T08:10:04.976ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208532025-11-13T08:10:04.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208542025-11-13T08:10:04.977ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208552025-11-13T08:10:04.980ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
208562025-11-13T08:10:04.980ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208572025-11-13T08:10:04.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208582025-11-13T08:10:04.982ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208592025-11-13T08:10:04.984ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
208602025-11-13T08:10:04.984ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208612025-11-13T08:10:04.985ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208622025-11-13T08:10:04.987ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208632025-11-13T08:10:04.987ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208642025-11-13T08:10:04.989ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208652025-11-13T08:10:04.989ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208662025-11-13T08:10:04.991ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208672025-11-13T08:10:04.994ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
208682025-11-13T08:10:04.994ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208692025-11-13T08:10:04.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208702025-11-13T08:10:04.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208712025-11-13T08:10:04.998ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
208722025-11-13T08:10:04.998ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208732025-11-13T08:10:04.998ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208742025-11-13T08:10:05.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208752025-11-13T08:10:05.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208762025-11-13T08:10:05.001ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208772025-11-13T08:10:05.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208782025-11-13T08:10:05.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208792025-11-13T08:10:05.005ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
208802025-11-13T08:10:05.006ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208812025-11-13T08:10:05.006ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208822025-11-13T08:10:05.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208832025-11-13T08:10:05.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208842025-11-13T08:10:05.008ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208852025-11-13T08:10:05.011ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
208862025-11-13T08:10:05.012ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208872025-11-13T08:10:05.012ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208882025-11-13T08:10:05.014ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208892025-11-13T08:10:05.015ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208902025-11-13T08:10:05.016ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208912025-11-13T08:10:05.019ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
208922025-11-13T08:10:05.019ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208932025-11-13T08:10:05.019ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208942025-11-13T08:10:05.020ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208952025-11-13T08:10:05.021ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
208962025-11-13T08:10:05.022ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
208972025-11-13T08:10:05.025ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
208982025-11-13T08:10:05.025ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
208992025-11-13T08:10:05.025ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
209002025-11-13T08:10:05.027ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
209012025-11-13T08:10:05.027ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
209022025-11-13T08:10:05.029ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209032025-11-13T08:10:05.031ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
209042025-11-13T08:10:05.032ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
209052025-11-13T08:10:05.032ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
209062025-11-13T08:10:05.034ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
209072025-11-13T08:10:05.034ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
209082025-11-13T08:10:05.035ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209092025-11-13T08:10:05.037ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
209102025-11-13T08:10:05.037ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
209112025-11-13T08:10:05.037ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
209122025-11-13T08:10:05.072ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
209132025-11-13T08:10:05.072ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
209142025-11-13T08:10:05.072ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 7cb04c89-8391-4444-ac6f-ef060e571df8 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
209152025-11-13T08:10:05.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
209162025-11-13T08:10:05.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
209172025-11-13T08:10:05.074ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209182025-11-13T08:10:05.077ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
209192025-11-13T08:10:05.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
209202025-11-13T08:10:05.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
209212025-11-13T08:10:05.079ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 1 limit = 5000 usage_percent = 0
209222025-11-13T08:10:05.079ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
209232025-11-13T08:10:05.079ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 213.969386ms iteration = 29 status = {"Unchanged":{"blueprint_count":1,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
209242025-11-13T08:10:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 440 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = e1f9b8d8-9211-4b69-b60b-92fc4c5419f6 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
209252025-11-13T08:10:07.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 134 reason = Timeout
209262025-11-13T08:10:07.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
209272025-11-13T08:10:07.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
209282025-11-13T08:10:07.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209292025-11-13T08:10:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
209302025-11-13T08:10:07.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
209312025-11-13T08:10:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
209322025-11-13T08:10:07.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
209332025-11-13T08:10:07.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.222331ms iteration = 134 status = {"config_updated":false}
209342025-11-13T08:10:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 68 reason = Timeout
209352025-11-13T08:10:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 135 reason = Timeout
209362025-11-13T08:10:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
209372025-11-13T08:10:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
209382025-11-13T08:10:12.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
209392025-11-13T08:10:12.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
209402025-11-13T08:10:12.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209412025-11-13T08:10:12.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209422025-11-13T08:10:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
209432025-11-13T08:10:12.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
209442025-11-13T08:10:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
209452025-11-13T08:10:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
209462025-11-13T08:10:12.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
209472025-11-13T08:10:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
209482025-11-13T08:10:12.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
209492025-11-13T08:10:12.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
209502025-11-13T08:10:12.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
209512025-11-13T08:10:12.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.219452ms iteration = 135 status = {"config_updated":false}
209522025-11-13T08:10:12.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209532025-11-13T08:10:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
209542025-11-13T08:10:12.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
209552025-11-13T08:10:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
209562025-11-13T08:10:12.641ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 68 reason = Timeout
209572025-11-13T08:10:12.641ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
209582025-11-13T08:10:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
209592025-11-13T08:10:12.654ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
209602025-11-13T08:10:12.654ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 28.361467ms iteration = 68 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
209612025-11-13T08:10:12.672ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
209622025-11-13T08:10:12.672ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
209632025-11-13T08:10:12.706ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.908766ms iteration = 68 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
209642025-11-13T08:10:15.473ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:64937
209652025-11-13T08:10:15.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 5e20cfaf-216b-4912-96df-5dcd7fe51110 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
209662025-11-13T08:10:15.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 5e20cfaf-216b-4912-96df-5dcd7fe51110 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
209672025-11-13T08:10:15.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 5e20cfaf-216b-4912-96df-5dcd7fe51110 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
209682025-11-13T08:10:15.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 3487 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 5e20cfaf-216b-4912-96df-5dcd7fe51110 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
209692025-11-13T08:10:15.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 03dad54c-9252-4cdc-80dc-16c80265b23b uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
209702025-11-13T08:10:15.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 03dad54c-9252-4cdc-80dc-16c80265b23b roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
209712025-11-13T08:10:15.477ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 03dad54c-9252-4cdc-80dc-16c80265b23b resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
209722025-11-13T08:10:15.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2617 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:64937 req_id = 03dad54c-9252-4cdc-80dc-16c80265b23b response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
209732025-11-13T08:10:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 264 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 77209080-8f66-4a17-b322-89a0e6453b8c response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
209742025-11-13T08:10:17.625ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 60 reason = Timeout
209752025-11-13T08:10:17.625ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 136 reason = Timeout
209762025-11-13T08:10:17.625ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
209772025-11-13T08:10:17.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
209782025-11-13T08:10:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
209792025-11-13T08:10:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
209802025-11-13T08:10:17.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209812025-11-13T08:10:17.627ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
209822025-11-13T08:10:17.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
209832025-11-13T08:10:17.630ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
209842025-11-13T08:10:17.630ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
209852025-11-13T08:10:17.630ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
209862025-11-13T08:10:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
209872025-11-13T08:10:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
209882025-11-13T08:10:17.632ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a7967a9a-1e1f-42c0-b592-9f2752cb200d original_time_started = 2025-11-13 08:10:04.244650 UTC
209892025-11-13T08:10:17.632ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.049989ms iteration = 60 status = {"Loaded":{"collection_id":"a7967a9a-1e1f-42c0-b592-9f2752cb200d","time_loaded":"2025-11-13T08:10:17.625918842Z","time_started":"2025-11-13T08:10:04.244650Z"}}
209902025-11-13T08:10:17.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
209912025-11-13T08:10:17.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.062219ms iteration = 136 status = {"config_updated":false}
209922025-11-13T08:10:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 46 reason = Timeout
209932025-11-13T08:10:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 46 reason = Timeout
209942025-11-13T08:10:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
209952025-11-13T08:10:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
209962025-11-13T08:10:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
209972025-11-13T08:10:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
209982025-11-13T08:10:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
209992025-11-13T08:10:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
210002025-11-13T08:10:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210012025-11-13T08:10:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210022025-11-13T08:10:17.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
210032025-11-13T08:10:17.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
210042025-11-13T08:10:17.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
210052025-11-13T08:10:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
210062025-11-13T08:10:17.653ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.702232ms iteration = 46 status = "NoSitrep"
210072025-11-13T08:10:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
210082025-11-13T08:10:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
210092025-11-13T08:10:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
210102025-11-13T08:10:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
210112025-11-13T08:10:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210122025-11-13T08:10:17.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
210132025-11-13T08:10:17.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
210142025-11-13T08:10:17.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.780052ms iteration = 46 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
210152025-11-13T08:10:22.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 69 reason = Timeout
210162025-11-13T08:10:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 137 reason = Timeout
210172025-11-13T08:10:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
210182025-11-13T08:10:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
210192025-11-13T08:10:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
210202025-11-13T08:10:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
210212025-11-13T08:10:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210222025-11-13T08:10:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210232025-11-13T08:10:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
210242025-11-13T08:10:22.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
210252025-11-13T08:10:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
210262025-11-13T08:10:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
210272025-11-13T08:10:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
210282025-11-13T08:10:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
210292025-11-13T08:10:22.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
210302025-11-13T08:10:22.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.60567ms iteration = 137 status = {"config_updated":false}
210312025-11-13T08:10:22.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
210322025-11-13T08:10:22.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
210332025-11-13T08:10:22.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210342025-11-13T08:10:22.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
210352025-11-13T08:10:22.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
210362025-11-13T08:10:22.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
210372025-11-13T08:10:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 69 reason = Timeout
210382025-11-13T08:10:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
210392025-11-13T08:10:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
210402025-11-13T08:10:22.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
210412025-11-13T08:10:22.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
210422025-11-13T08:10:22.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.945558ms iteration = 69 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
210432025-11-13T08:10:22.650ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
210442025-11-13T08:10:22.650ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.303388ms iteration = 69 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
210452025-11-13T08:10:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 455 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 0a995721-61df-4230-a340-6e28ee9c5a7e response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
210462025-11-13T08:10:27.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 138 reason = Timeout
210472025-11-13T08:10:27.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
210482025-11-13T08:10:27.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
210492025-11-13T08:10:27.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210502025-11-13T08:10:27.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
210512025-11-13T08:10:27.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
210522025-11-13T08:10:27.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
210532025-11-13T08:10:27.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
210542025-11-13T08:10:27.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.64304ms iteration = 138 status = {"config_updated":false}
210552025-11-13T08:10:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 24 reason = Timeout
210562025-11-13T08:10:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 24 reason = Timeout
210572025-11-13T08:10:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 24 reason = Timeout
210582025-11-13T08:10:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
210592025-11-13T08:10:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
210602025-11-13T08:10:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
210612025-11-13T08:10:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
210622025-11-13T08:10:32.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6640, tv_nsec: 963584956 } })
210632025-11-13T08:10:32.612ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6640, tv_nsec: 964375861 } })
210642025-11-13T08:10:32.612ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
210652025-11-13T08:10:32.614ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.57023ms iteration = 24 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
210662025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 36 reason = Timeout
210672025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
210682025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 61 reason = Timeout
210692025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 70 reason = Timeout
210702025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
210712025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 39 reason = Timeout
210722025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
210732025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 139 reason = Timeout
210742025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
210752025-11-13T08:10:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
210762025-11-13T08:10:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
210772025-11-13T08:10:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
210782025-11-13T08:10:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
210792025-11-13T08:10:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
210802025-11-13T08:10:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
210812025-11-13T08:10:32.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210822025-11-13T08:10:32.631ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210832025-11-13T08:10:32.634ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210842025-11-13T08:10:32.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210852025-11-13T08:10:32.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
210862025-11-13T08:10:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
210872025-11-13T08:10:32.635ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
210882025-11-13T08:10:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
210892025-11-13T08:10:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
210902025-11-13T08:10:32.637ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
210912025-11-13T08:10:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
210922025-11-13T08:10:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
210932025-11-13T08:10:32.638ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
210942025-11-13T08:10:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
210952025-11-13T08:10:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
210962025-11-13T08:10:32.638ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
210972025-11-13T08:10:32.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
210982025-11-13T08:10:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
210992025-11-13T08:10:32.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211002025-11-13T08:10:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
211012025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 24 reason = Timeout
211022025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
211032025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 24 reason = Timeout
211042025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 36 reason = Timeout
211052025-11-13T08:10:32.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
211062025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 24 reason = Timeout
211072025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
211082025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
211092025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 24 reason = Timeout
211102025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
211112025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
211122025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 24 reason = Timeout
211132025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
211142025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
211152025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 24 reason = Timeout
211162025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
211172025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 70 reason = Timeout
211182025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
211192025-11-13T08:10:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
211202025-11-13T08:10:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
211212025-11-13T08:10:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
211222025-11-13T08:10:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
211232025-11-13T08:10:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
211242025-11-13T08:10:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
211252025-11-13T08:10:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
211262025-11-13T08:10:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
211272025-11-13T08:10:32.646ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
211282025-11-13T08:10:32.646ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 19.919662ms iteration = 139 status = {"config_updated":false}
211292025-11-13T08:10:32.646ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
211302025-11-13T08:10:32.646ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
211312025-11-13T08:10:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
211322025-11-13T08:10:32.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
211332025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 24 reason = Timeout
211342025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 24 reason = Timeout
211352025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
211362025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
211372025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 47 reason = Timeout
211382025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 47 reason = Timeout
211392025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
211402025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211412025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 24 reason = Timeout
211422025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 24 reason = Timeout
211432025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
211442025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 24 reason = Timeout
211452025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
211462025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 24 reason = Timeout
211472025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
211482025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
211492025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
211502025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211512025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
211522025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
211532025-11-13T08:10:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
211542025-11-13T08:10:32.648ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6641, tv_nsec: 1006561 } })
211552025-11-13T08:10:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
211562025-11-13T08:10:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211572025-11-13T08:10:32.649ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211582025-11-13T08:10:32.650ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a7967a9a-1e1f-42c0-b592-9f2752cb200d original_time_started = 2025-11-13 08:10:04.244650 UTC
211592025-11-13T08:10:32.650ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 23.980391ms iteration = 61 status = {"Loaded":{"collection_id":"a7967a9a-1e1f-42c0-b592-9f2752cb200d","time_loaded":"2025-11-13T08:10:32.626463436Z","time_started":"2025-11-13T08:10:04.244650Z"}}
211602025-11-13T08:10:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211612025-11-13T08:10:32.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
211622025-11-13T08:10:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
211632025-11-13T08:10:32.653ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
211642025-11-13T08:10:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
211652025-11-13T08:10:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
211662025-11-13T08:10:32.654ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
211672025-11-13T08:10:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
211682025-11-13T08:10:32.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211692025-11-13T08:10:32.657ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
211702025-11-13T08:10:32.658ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 15.566946ms iteration = 24 status = 0
211712025-11-13T08:10:32.662ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211722025-11-13T08:10:32.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211732025-11-13T08:10:32.663ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
211742025-11-13T08:10:32.664ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
211752025-11-13T08:10:32.666ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211762025-11-13T08:10:32.669ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
211772025-11-13T08:10:32.670ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
211782025-11-13T08:10:32.670ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 22.110645ms iteration = 47 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
211792025-11-13T08:10:32.702ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
211802025-11-13T08:10:32.702ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
211812025-11-13T08:10:32.706ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2e43658-ef63-4405-89c2-3a3ddd5ca444", "content-length": "26", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
211822025-11-13T08:10:32.706ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
211832025-11-13T08:10:32.707ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
211842025-11-13T08:10:32.707ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
211852025-11-13T08:10:32.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
211862025-11-13T08:10:32.716ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
211872025-11-13T08:10:32.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
211882025-11-13T08:10:32.717ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 69.377634ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
211892025-11-13T08:10:32.719ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
211902025-11-13T08:10:32.719ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
211912025-11-13T08:10:32.720ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "5047d137-a399-4f8a-a7b5-59dbcf48c44a", "content-length": "78", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
211922025-11-13T08:10:32.720ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
211932025-11-13T08:10:32.720ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
211942025-11-13T08:10:32.720ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
211952025-11-13T08:10:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
211962025-11-13T08:10:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
211972025-11-13T08:10:32.721ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
211982025-11-13T08:10:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
211992025-11-13T08:10:32.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
212002025-11-13T08:10:32.723ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
212012025-11-13T08:10:32.723ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
212022025-11-13T08:10:32.723ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
212032025-11-13T08:10:32.723ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
212042025-11-13T08:10:32.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
212052025-11-13T08:10:32.723ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
212062025-11-13T08:10:32.726ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212072025-11-13T08:10:32.728ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212082025-11-13T08:10:32.728ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
212092025-11-13T08:10:32.728ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
212102025-11-13T08:10:32.728ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
212112025-11-13T08:10:32.728ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
212122025-11-13T08:10:32.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
212132025-11-13T08:10:32.729ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
212142025-11-13T08:10:32.729ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
212152025-11-13T08:10:32.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
212162025-11-13T08:10:32.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
212172025-11-13T08:10:32.730ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 82.035672ms iteration = 24 status = {"errors":[],"finish_invoked_ok":[]}
212182025-11-13T08:10:32.730ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
212192025-11-13T08:10:32.730ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
212202025-11-13T08:10:32.731ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212212025-11-13T08:10:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
212222025-11-13T08:10:32.732ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
212232025-11-13T08:10:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
212242025-11-13T08:10:32.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
212252025-11-13T08:10:32.733ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212262025-11-13T08:10:32.734ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
212272025-11-13T08:10:32.734ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
212282025-11-13T08:10:32.734ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
212292025-11-13T08:10:32.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
212302025-11-13T08:10:32.735ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
212312025-11-13T08:10:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
212322025-11-13T08:10:32.735ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
212332025-11-13T08:10:32.736ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 87.756075ms iteration = 24 status = {"errors":[],"garbage_collect_requested":[]}
212342025-11-13T08:10:32.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
212352025-11-13T08:10:32.737ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 129.495046ms iteration = 24 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
212362025-11-13T08:10:32.738ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6641, tv_nsec: 90461300 } })
212372025-11-13T08:10:32.738ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
212382025-11-13T08:10:32.740ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
212392025-11-13T08:10:32.740ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
212402025-11-13T08:10:32.740ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
212412025-11-13T08:10:32.741ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
212422025-11-13T08:10:32.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
212432025-11-13T08:10:32.744ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212442025-11-13T08:10:32.744ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212452025-11-13T08:10:32.745ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 97.406285ms iteration = 24 status = {"errors":[],"requests_created_ok":[]}
212462025-11-13T08:10:32.748ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
212472025-11-13T08:10:32.748ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
212482025-11-13T08:10:32.748ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
212492025-11-13T08:10:32.749ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
212502025-11-13T08:10:32.749ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
212512025-11-13T08:10:32.749ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
212522025-11-13T08:10:32.750ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 102.807161ms iteration = 47 status = "NoSitrep"
212532025-11-13T08:10:32.764ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
212542025-11-13T08:10:32.764ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 137.993553ms iteration = 70 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
212552025-11-13T08:10:32.768ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 125.823933ms iteration = 70 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
212562025-11-13T08:10:32.776ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 168.730518ms iteration = 24 status = {}
212572025-11-13T08:10:32.786ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
212582025-11-13T08:10:32.787ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
212592025-11-13T08:10:32.787ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "2867bc64-a7b4-4faa-8cc1-4fa8a1b2c444", "content-length": "81", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
212602025-11-13T08:10:32.787ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
212612025-11-13T08:10:32.787ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
212622025-11-13T08:10:32.788ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "c570ddcf-6b39-4fb4-87c7-ef38e9889b2b", "content-length": "81", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
212632025-11-13T08:10:32.790ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
212642025-11-13T08:10:32.790ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 164.425652ms iteration = 36 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
212652025-11-13T08:10:32.797ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
212662025-11-13T08:10:32.797ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
212672025-11-13T08:10:32.797ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb3c7745-4512-402f-9434-4a09a8e37894", "content-length": "81", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
212682025-11-13T08:10:32.799ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
212692025-11-13T08:10:32.799ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
212702025-11-13T08:10:32.800ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b97e3199-3754-4cae-90d4-4c8ede1ff9b5", "content-length": "26", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
212712025-11-13T08:10:32.800ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
212722025-11-13T08:10:32.800ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 151.913487ms iteration = 24 status = {"disabled":false,"errors":[],"sps":[]}
212732025-11-13T08:10:32.800ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 157.987916ms iteration = 24 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
212742025-11-13T08:10:32.801ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
212752025-11-13T08:10:32.802ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
212762025-11-13T08:10:32.802ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
212772025-11-13T08:10:32.802ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
212782025-11-13T08:10:32.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
212792025-11-13T08:10:32.812ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
212802025-11-13T08:10:32.812ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
212812025-11-13T08:10:32.812ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 170.215827ms iteration = 24 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
212822025-11-13T08:10:32.813ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
212832025-11-13T08:10:32.816ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212842025-11-13T08:10:32.818ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
212852025-11-13T08:10:32.818ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
212862025-11-13T08:10:32.819ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
212872025-11-13T08:10:32.854ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6641, tv_nsec: 206775666 } })
212882025-11-13T08:10:32.860ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
212892025-11-13T08:10:32.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
212902025-11-13T08:10:32.862ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
212912025-11-13T08:10:32.862ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
212922025-11-13T08:10:32.862ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 219.328072ms iteration = 24 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
212932025-11-13T08:10:32.869ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 221.886862ms iteration = 24 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
212942025-11-13T08:10:32.910ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 284.083105ms iteration = 39 status = {"physical_disks_added":0}
212952025-11-13T08:10:32.925ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
212962025-11-13T08:10:32.926ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
212972025-11-13T08:10:32.927ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
212982025-11-13T08:10:32.930ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
212992025-11-13T08:10:32.930ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
213002025-11-13T08:10:32.930ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
213012025-11-13T08:10:32.932ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213022025-11-13T08:10:32.935ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
213032025-11-13T08:10:32.935ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
213042025-11-13T08:10:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
213052025-11-13T08:10:32.946ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
213062025-11-13T08:10:32.946ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
213072025-11-13T08:10:32.950ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
213082025-11-13T08:10:32.950ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
213092025-11-13T08:10:32.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
213102025-11-13T08:10:32.964ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
213112025-11-13T08:10:32.964ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
213122025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
213132025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
213142025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
213152025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
213162025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
213172025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
213182025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
213192025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
213202025-11-13T08:10:32.965ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
213212025-11-13T08:10:32.965ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
213222025-11-13T08:10:32.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
213232025-11-13T08:10:32.968ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
213242025-11-13T08:10:32.968ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
213252025-11-13T08:10:32.976ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
213262025-11-13T08:10:32.976ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
213272025-11-13T08:10:32.976ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3ec84b2-f00d-44e0-87d0-8d715015d5a2", "content-length": "372", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
213282025-11-13T08:10:32.976ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
213292025-11-13T08:10:32.976ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
213302025-11-13T08:10:32.983ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
213312025-11-13T08:10:32.983ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5607903b-0aeb-45fd-9546-6529e9009cec", "content-length": "691", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
213322025-11-13T08:10:32.983ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
213332025-11-13T08:10:32.983ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
213342025-11-13T08:10:32.983ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
213352025-11-13T08:10:32.984ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 342.034094ms iteration = 24 status = {}
213362025-11-13T08:10:32.990ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
213372025-11-13T08:10:32.990ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
213382025-11-13T08:10:32.991ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "7e3060cb-7237-4d7f-97fe-04b3617ed82e", "date": "Thu, 13 Nov 2025 08:10:32 GMT"} })
213392025-11-13T08:10:32.991ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
213402025-11-13T08:10:32.991ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
213412025-11-13T08:10:32.994ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
213422025-11-13T08:10:32.994ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
213432025-11-13T08:10:32.997ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
213442025-11-13T08:10:32.997ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
213452025-11-13T08:10:33.000ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
213462025-11-13T08:10:33.000ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
213472025-11-13T08:10:33.005ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "40eb9b30-c8aa-4ffe-9968-35671499a96b", "date": "Thu, 13 Nov 2025 08:10:33 GMT"} })
213482025-11-13T08:10:33.006ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 364.117688ms iteration = 36 status = {}
213492025-11-13T08:10:33.307ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "b7316088-01bc-451d-8620-8ae0ec414c06", "date": "Thu, 13 Nov 2025 08:10:33 GMT"} })
213502025-11-13T08:10:33.307ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
213512025-11-13T08:10:33.307ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
213522025-11-13T08:10:33.307ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb4ba55a-0f99-4ca4-ba4c-cde4b8a10e25", "content-length": "682", "date": "Thu, 13 Nov 2025 08:10:33 GMT"} })
213532025-11-13T08:10:33.307ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
213542025-11-13T08:10:33.308ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
213552025-11-13T08:10:33.311ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
213562025-11-13T08:10:33.312ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
213572025-11-13T08:10:33.315ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
213582025-11-13T08:10:33.315ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
213592025-11-13T08:10:33.317ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
213602025-11-13T08:10:33.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
213612025-11-13T08:10:33.324ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 682.055649ms iteration = 24 status = {}
213622025-11-13T08:10:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 329 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = bb2af04c-3f17-4411-8b0a-1caa352d71fd response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
213632025-11-13T08:10:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 140 reason = Timeout
213642025-11-13T08:10:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
213652025-11-13T08:10:37.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
213662025-11-13T08:10:37.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213672025-11-13T08:10:37.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
213682025-11-13T08:10:37.634ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
213692025-11-13T08:10:37.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
213702025-11-13T08:10:37.637ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
213712025-11-13T08:10:37.637ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.434655ms iteration = 140 status = {"config_updated":false}
213722025-11-13T08:10:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 71 reason = Timeout
213732025-11-13T08:10:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 141 reason = Timeout
213742025-11-13T08:10:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
213752025-11-13T08:10:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
213762025-11-13T08:10:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
213772025-11-13T08:10:42.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
213782025-11-13T08:10:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213792025-11-13T08:10:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213802025-11-13T08:10:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
213812025-11-13T08:10:42.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
213822025-11-13T08:10:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
213832025-11-13T08:10:42.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
213842025-11-13T08:10:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
213852025-11-13T08:10:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
213862025-11-13T08:10:42.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
213872025-11-13T08:10:42.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.211802ms iteration = 141 status = {"config_updated":false}
213882025-11-13T08:10:42.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
213892025-11-13T08:10:42.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
213902025-11-13T08:10:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
213912025-11-13T08:10:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
213922025-11-13T08:10:42.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
213932025-11-13T08:10:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
213942025-11-13T08:10:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 71 reason = Timeout
213952025-11-13T08:10:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
213962025-11-13T08:10:42.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
213972025-11-13T08:10:42.650ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
213982025-11-13T08:10:42.650ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.780047ms iteration = 71 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
213992025-11-13T08:10:42.683ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
214002025-11-13T08:10:42.683ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
214012025-11-13T08:10:42.718ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 75.331932ms iteration = 71 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
214022025-11-13T08:10:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 385 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = a5ac272e-b4b0-4d50-b59f-c609288036a4 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
214032025-11-13T08:10:47.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 62 reason = Timeout
214042025-11-13T08:10:47.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 142 reason = Timeout
214052025-11-13T08:10:47.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
214062025-11-13T08:10:47.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
214072025-11-13T08:10:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
214082025-11-13T08:10:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
214092025-11-13T08:10:47.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214102025-11-13T08:10:47.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214112025-11-13T08:10:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
214122025-11-13T08:10:47.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
214132025-11-13T08:10:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
214142025-11-13T08:10:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
214152025-11-13T08:10:47.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
214162025-11-13T08:10:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
214172025-11-13T08:10:47.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a7967a9a-1e1f-42c0-b592-9f2752cb200d original_time_started = 2025-11-13 08:10:04.244650 UTC
214182025-11-13T08:10:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.168451ms iteration = 62 status = {"Loaded":{"collection_id":"a7967a9a-1e1f-42c0-b592-9f2752cb200d","time_loaded":"2025-11-13T08:10:47.626315482Z","time_started":"2025-11-13T08:10:04.244650Z"}}
214192025-11-13T08:10:47.635ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
214202025-11-13T08:10:47.635ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 9.062489ms iteration = 142 status = {"config_updated":false}
214212025-11-13T08:10:47.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 48 reason = Timeout
214222025-11-13T08:10:47.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
214232025-11-13T08:10:47.647ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 48 reason = Timeout
214242025-11-13T08:10:47.647ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
214252025-11-13T08:10:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
214262025-11-13T08:10:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
214272025-11-13T08:10:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
214282025-11-13T08:10:47.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
214292025-11-13T08:10:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214302025-11-13T08:10:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214312025-11-13T08:10:47.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
214322025-11-13T08:10:47.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
214332025-11-13T08:10:47.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
214342025-11-13T08:10:47.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
214352025-11-13T08:10:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
214362025-11-13T08:10:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
214372025-11-13T08:10:47.655ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 7.433056ms iteration = 48 status = "NoSitrep"
214382025-11-13T08:10:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
214392025-11-13T08:10:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
214402025-11-13T08:10:47.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214412025-11-13T08:10:47.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
214422025-11-13T08:10:47.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
214432025-11-13T08:10:47.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.339583ms iteration = 48 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
214442025-11-13T08:10:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 72 reason = Timeout
214452025-11-13T08:10:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 143 reason = Timeout
214462025-11-13T08:10:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
214472025-11-13T08:10:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
214482025-11-13T08:10:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
214492025-11-13T08:10:52.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
214502025-11-13T08:10:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214512025-11-13T08:10:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214522025-11-13T08:10:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
214532025-11-13T08:10:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
214542025-11-13T08:10:52.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
214552025-11-13T08:10:52.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
214562025-11-13T08:10:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
214572025-11-13T08:10:52.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
214582025-11-13T08:10:52.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
214592025-11-13T08:10:52.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
214602025-11-13T08:10:52.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.959407ms iteration = 143 status = {"config_updated":false}
214612025-11-13T08:10:52.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
214622025-11-13T08:10:52.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214632025-11-13T08:10:52.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
214642025-11-13T08:10:52.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
214652025-11-13T08:10:52.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
214662025-11-13T08:10:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 72 reason = Timeout
214672025-11-13T08:10:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
214682025-11-13T08:10:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
214692025-11-13T08:10:52.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
214702025-11-13T08:10:52.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
214712025-11-13T08:10:52.645ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.866907ms iteration = 72 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
214722025-11-13T08:10:52.649ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
214732025-11-13T08:10:52.649ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.311138ms iteration = 72 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
214742025-11-13T08:10:55.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 311 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 49282cb1-43c2-4a41-93f2-47bf74789e61 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
214752025-11-13T08:10:57.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 144 reason = Timeout
214762025-11-13T08:10:57.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
214772025-11-13T08:10:57.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
214782025-11-13T08:10:57.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
214792025-11-13T08:10:57.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
214802025-11-13T08:10:57.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
214812025-11-13T08:10:57.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
214822025-11-13T08:10:57.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
214832025-11-13T08:10:57.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.261513ms iteration = 144 status = {"config_updated":false}
214842025-11-13T08:11:00.473ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::d]:57512
214852025-11-13T08:11:00.473ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = acc6fdc3-0590-46e6-b72b-a936f328f684 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending version = 1.0.0
214862025-11-13T08:11:00.473ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = acc6fdc3-0590-46e6-b72b-a936f328f684 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
214872025-11-13T08:11:00.473ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = acc6fdc3-0590-46e6-b72b-a936f328f684 resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending user_builtin_id = 001de000-05e4-4000-8000-000000000002
214882025-11-13T08:11:00.476ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2742 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = acc6fdc3-0590-46e6-b72b-a936f328f684 response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?sort_by=id_ascending
214892025-11-13T08:11:00.476ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = cff16700-d086-488b-b948-5828cc6133cd uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D version = 1.0.0
214902025-11-13T08:11:00.476ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = cff16700-d086-488b-b948-5828cc6133cd roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
214912025-11-13T08:11:00.476ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = cff16700-d086-488b-b948-5828cc6133cd resource = Database result = Ok(()) type = user_builtin uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D user_builtin_id = 001de000-05e4-4000-8000-000000000002
214922025-11-13T08:11:00.479ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 2740 local_addr = [fd00:1122:3344:101::a]:12221 method = GET remote_addr = [fd00:1122:3344:101::d]:57512 req_id = cff16700-d086-488b-b948-5828cc6133cd response_code = 200 uri = /metrics/collectors/87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmMmM0Y2QyOS0wZmJmLTQ2NDgtOTUzZi1mOGI4ODkzNGViYzEifX0%3D
214932025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 13 reason = Timeout
214942025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 13 reason = Timeout
214952025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 13 reason = Timeout
214962025-11-13T08:11:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 08:01:02.607143784 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
214972025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
214982025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 25 reason = Timeout
214992025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
215002025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 15 reason = Timeout
215012025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 25 reason = Timeout
215022025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 25 reason = Timeout
215032025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
215042025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 13 reason = Timeout
215052025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
215062025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
215072025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 15 reason = Timeout
215082025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 13 reason = Timeout
215092025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
215102025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 13 reason = Timeout
215112025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
215122025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
215132025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
215142025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
215152025-11-13T08:11:02.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
215162025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 14 reason = Timeout
215172025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
215182025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
215192025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
215202025-11-13T08:11:02.608ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
215212025-11-13T08:11:02.608ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = ExternalNetworking step id = Ensure
215222025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
215232025-11-13T08:11:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6670, tv_nsec: 960318598 } })
215242025-11-13T08:11:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6670, tv_nsec: 960332209 } })
215252025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
215262025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
215272025-11-13T08:11:02.608ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
215282025-11-13T08:11:02.609ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
215292025-11-13T08:11:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.955939ms iteration = 13 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
215302025-11-13T08:11:02.609ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
215312025-11-13T08:11:02.609ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.880687ms iteration = 13 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
215322025-11-13T08:11:02.609ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
215332025-11-13T08:11:02.611ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
215342025-11-13T08:11:02.611ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215352025-11-13T08:11:02.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
215362025-11-13T08:11:02.612ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 08:01:02.607143784 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
215372025-11-13T08:11:02.612ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 5.107051ms iteration = 13 status = {"expiration":"2025-11-13T08:01:02.607143784Z","pruned":[]}
215382025-11-13T08:11:02.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
215392025-11-13T08:11:02.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
215402025-11-13T08:11:02.614ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
215412025-11-13T08:11:02.614ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6670, tv_nsec: 966737715 } })
215422025-11-13T08:11:02.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
215432025-11-13T08:11:02.616ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
215442025-11-13T08:11:02.617ZDEBGnexus (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 6 reason = Timeout
215452025-11-13T08:11:02.617ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 13 reason = Timeout
215462025-11-13T08:11:02.617ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
215472025-11-13T08:11:02.617ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
215482025-11-13T08:11:02.617ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
215492025-11-13T08:11:02.617ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 30 reason = Timeout
215502025-11-13T08:11:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
215512025-11-13T08:11:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
215522025-11-13T08:11:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
215532025-11-13T08:11:02.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
215542025-11-13T08:11:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
215552025-11-13T08:11:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
215562025-11-13T08:11:02.618ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
215572025-11-13T08:11:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
215582025-11-13T08:11:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
215592025-11-13T08:11:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
215602025-11-13T08:11:02.619ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
215612025-11-13T08:11:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
215622025-11-13T08:11:02.620ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6670, tv_nsec: 972352255 } })
215632025-11-13T08:11:02.620ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
215642025-11-13T08:11:02.621ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
215652025-11-13T08:11:02.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
215662025-11-13T08:11:02.623ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215672025-11-13T08:11:02.624ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215682025-11-13T08:11:02.625ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
215692025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
215702025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 37 reason = Timeout
215712025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 63 reason = Timeout
215722025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 73 reason = Timeout
215732025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
215742025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
215752025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
215762025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 145 reason = Timeout
215772025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
215782025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 40 reason = Timeout
215792025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
215802025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215812025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
215822025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
215832025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
215842025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
215852025-11-13T08:11:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
215862025-11-13T08:11:02.628ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215872025-11-13T08:11:02.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215882025-11-13T08:11:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
215892025-11-13T08:11:02.628ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
215902025-11-13T08:11:02.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
215912025-11-13T08:11:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
215922025-11-13T08:11:02.629ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
215932025-11-13T08:11:02.629ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
215942025-11-13T08:11:02.629ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215952025-11-13T08:11:02.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215962025-11-13T08:11:02.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215972025-11-13T08:11:02.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
215982025-11-13T08:11:02.630ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 23.342219ms iteration = 25 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
215992025-11-13T08:11:02.634ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
216002025-11-13T08:11:02.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216012025-11-13T08:11:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216022025-11-13T08:11:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
216032025-11-13T08:11:02.635ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
216042025-11-13T08:11:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
216052025-11-13T08:11:02.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
216062025-11-13T08:11:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
216072025-11-13T08:11:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
216082025-11-13T08:11:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
216092025-11-13T08:11:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
216102025-11-13T08:11:02.636ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
216112025-11-13T08:11:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
216122025-11-13T08:11:02.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216132025-11-13T08:11:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
216142025-11-13T08:11:02.637ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
216152025-11-13T08:11:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
216162025-11-13T08:11:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
216172025-11-13T08:11:02.640ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216182025-11-13T08:11:02.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
216192025-11-13T08:11:02.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
216202025-11-13T08:11:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
216212025-11-13T08:11:02.641ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
216222025-11-13T08:11:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 13 reason = Timeout
216232025-11-13T08:11:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
216242025-11-13T08:11:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
216252025-11-13T08:11:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 13 reason = Timeout
216262025-11-13T08:11:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
216272025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
216282025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 25 reason = Timeout
216292025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
216302025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 25 reason = Timeout
216312025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 37 reason = Timeout
216322025-11-13T08:11:02.643ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
216332025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 25 reason = Timeout
216342025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
216352025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
216362025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
216372025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 73 reason = Timeout
216382025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
216392025-11-13T08:11:02.643ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 25 reason = Timeout
216402025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
216412025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
216422025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 25 reason = Timeout
216432025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 25 reason = Timeout
216442025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
216452025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
216462025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
216472025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
216482025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
216492025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
216502025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
216512025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
216522025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
216532025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 13 reason = Timeout
216542025-11-13T08:11:02.644ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
216552025-11-13T08:11:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
216562025-11-13T08:11:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
216572025-11-13T08:11:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
216582025-11-13T08:11:02.646ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
216592025-11-13T08:11:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
216602025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 13 reason = Timeout
216612025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
216622025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 25 reason = Timeout
216632025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
216642025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 25 reason = Timeout
216652025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
216662025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
216672025-11-13T08:11:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 25 reason = Timeout
216682025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
216692025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
216702025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 13 reason = Timeout
216712025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
216722025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 49 reason = Timeout
216732025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 25 reason = Timeout
216742025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
216752025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
216762025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 49 reason = Timeout
216772025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 13 reason = Timeout
216782025-11-13T08:11:02.655ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
216792025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
216802025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
216812025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 25 reason = Timeout
216822025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
216832025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 25 reason = Timeout
216842025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
216852025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
216862025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
216872025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
216882025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 23.207926ms iteration = 145 status = {"config_updated":false}
216892025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
216902025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
216912025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
216922025-11-13T08:11:02.655ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6671, tv_nsec: 1785944 } })
216932025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
216942025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
216952025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
216962025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
216972025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
216982025-11-13T08:11:02.655ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
216992025-11-13T08:11:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
217002025-11-13T08:11:02.657ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217012025-11-13T08:11:02.657ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
217022025-11-13T08:11:02.657ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
217032025-11-13T08:11:02.660ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217042025-11-13T08:11:02.665ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
217052025-11-13T08:11:02.665ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
217062025-11-13T08:11:02.665ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
217072025-11-13T08:11:02.667ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217082025-11-13T08:11:02.669ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
217092025-11-13T08:11:02.670ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 25.266707ms iteration = 13 status = {"errors":[],"found_port_ok":[]}
217102025-11-13T08:11:02.670ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
217112025-11-13T08:11:02.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
217122025-11-13T08:11:02.672ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
217132025-11-13T08:11:02.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
217142025-11-13T08:11:02.673ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
217152025-11-13T08:11:02.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
217162025-11-13T08:11:02.673ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
217172025-11-13T08:11:02.673ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
217182025-11-13T08:11:02.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
217192025-11-13T08:11:02.674ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
217202025-11-13T08:11:02.674ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 26.116694ms iteration = 13 status = {"by_rx":{},"error":null}
217212025-11-13T08:11:02.674ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
217222025-11-13T08:11:02.680ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
217232025-11-13T08:11:02.686ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
217242025-11-13T08:11:02.691ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
217252025-11-13T08:11:02.694ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
217262025-11-13T08:11:02.716ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
217272025-11-13T08:11:02.724ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
217282025-11-13T08:11:02.728ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
217292025-11-13T08:11:02.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217302025-11-13T08:11:02.732ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217312025-11-13T08:11:02.732ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
217322025-11-13T08:11:02.734ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
217332025-11-13T08:11:02.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
217342025-11-13T08:11:02.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
217352025-11-13T08:11:02.734ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217362025-11-13T08:11:02.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
217372025-11-13T08:11:02.736ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
217382025-11-13T08:11:02.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
217392025-11-13T08:11:02.738ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
217402025-11-13T08:11:02.738ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
217412025-11-13T08:11:02.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
217422025-11-13T08:11:02.744ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
217432025-11-13T08:11:02.745ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
217442025-11-13T08:11:02.749ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
217452025-11-13T08:11:02.749ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
217462025-11-13T08:11:02.749ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 107.426832ms iteration = 13 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
217472025-11-13T08:11:02.749ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217482025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
217492025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
217502025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
217512025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
217522025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
217532025-11-13T08:11:02.750ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
217542025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 132.355402ms iteration = 13 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
217552025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
217562025-11-13T08:11:02.750ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
217572025-11-13T08:11:02.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
217582025-11-13T08:11:02.751ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 102.921604ms iteration = 25 status = {"errors":[],"finish_invoked_ok":[]}
217592025-11-13T08:11:02.752ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = a7967a9a-1e1f-42c0-b592-9f2752cb200d original_time_started = 2025-11-13 08:10:04.244650 UTC
217602025-11-13T08:11:02.752ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 126.504087ms iteration = 63 status = {"Loaded":{"collection_id":"a7967a9a-1e1f-42c0-b592-9f2752cb200d","time_loaded":"2025-11-13T08:11:02.626148358Z","time_started":"2025-11-13T08:10:04.244650Z"}}
217612025-11-13T08:11:02.752ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217622025-11-13T08:11:02.753ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217632025-11-13T08:11:02.753ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
217642025-11-13T08:11:02.753ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
217652025-11-13T08:11:02.753ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
217662025-11-13T08:11:02.754ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = SledList step id = Fetch
217672025-11-13T08:11:02.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
217682025-11-13T08:11:02.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
217692025-11-13T08:11:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
217702025-11-13T08:11:02.754ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217712025-11-13T08:11:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
217722025-11-13T08:11:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
217732025-11-13T08:11:02.754ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217742025-11-13T08:11:02.756ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 107.579225ms iteration = 49 status = "NoSitrep"
217752025-11-13T08:11:02.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
217762025-11-13T08:11:02.757ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6671, tv_nsec: 109425970 } })
217772025-11-13T08:11:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
217782025-11-13T08:11:02.757ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
217792025-11-13T08:11:02.758ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217802025-11-13T08:11:02.758ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
217812025-11-13T08:11:02.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
217822025-11-13T08:11:02.758ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
217832025-11-13T08:11:02.758ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217842025-11-13T08:11:02.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
217852025-11-13T08:11:02.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
217862025-11-13T08:11:02.759ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217872025-11-13T08:11:02.759ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217882025-11-13T08:11:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
217892025-11-13T08:11:02.759ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
217902025-11-13T08:11:02.759ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
217912025-11-13T08:11:02.759ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
217922025-11-13T08:11:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
217932025-11-13T08:11:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
217942025-11-13T08:11:02.760ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
217952025-11-13T08:11:02.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217962025-11-13T08:11:02.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
217972025-11-13T08:11:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
217982025-11-13T08:11:02.763ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
217992025-11-13T08:11:02.763ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
218002025-11-13T08:11:02.763ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
218012025-11-13T08:11:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
218022025-11-13T08:11:02.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
218032025-11-13T08:11:02.763ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218042025-11-13T08:11:02.764ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
218052025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
218062025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
218072025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
218082025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
218092025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
218102025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
218112025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
218122025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
218132025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
218142025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
218152025-11-13T08:11:02.768ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218162025-11-13T08:11:02.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218172025-11-13T08:11:02.772ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
218182025-11-13T08:11:02.772ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
218192025-11-13T08:11:02.772ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
218202025-11-13T08:11:02.772ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
218212025-11-13T08:11:02.773ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
218222025-11-13T08:11:02.774ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 131.665389ms iteration = 25 status = 0
218232025-11-13T08:11:02.777ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218242025-11-13T08:11:02.778ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218252025-11-13T08:11:02.779ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 130.879393ms iteration = 13 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
218262025-11-13T08:11:02.780ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
218272025-11-13T08:11:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
218282025-11-13T08:11:02.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
218292025-11-13T08:11:02.781ZDEBGnexus (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
218302025-11-13T08:11:02.781ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
218312025-11-13T08:11:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
218322025-11-13T08:11:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
218332025-11-13T08:11:02.782ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
218342025-11-13T08:11:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
218352025-11-13T08:11:02.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
218362025-11-13T08:11:02.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218372025-11-13T08:11:02.786ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218382025-11-13T08:11:02.787ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
218392025-11-13T08:11:02.787ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
218402025-11-13T08:11:02.787ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
218412025-11-13T08:11:02.787ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
218422025-11-13T08:11:02.787ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
218432025-11-13T08:11:02.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
218442025-11-13T08:11:02.789ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
218452025-11-13T08:11:02.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
218462025-11-13T08:11:02.791ZDEBGnexus (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 173.034272ms iteration = 6 status = {"nsuccess":5}
218472025-11-13T08:11:02.791ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218482025-11-13T08:11:02.792ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218492025-11-13T08:11:02.792ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
218502025-11-13T08:11:02.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218512025-11-13T08:11:02.792ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
218522025-11-13T08:11:02.792ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
218532025-11-13T08:11:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218542025-11-13T08:11:02.793ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
218552025-11-13T08:11:02.793ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218562025-11-13T08:11:02.793ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
218572025-11-13T08:11:02.793ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
218582025-11-13T08:11:02.793ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
218592025-11-13T08:11:02.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
218602025-11-13T08:11:02.795ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
218612025-11-13T08:11:02.796ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
218622025-11-13T08:11:02.796ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
218632025-11-13T08:11:02.796ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
218642025-11-13T08:11:02.796ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
218652025-11-13T08:11:02.796ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 147.449759ms iteration = 49 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
218662025-11-13T08:11:02.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
218672025-11-13T08:11:02.796ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
218682025-11-13T08:11:02.796ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
218692025-11-13T08:11:02.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
218702025-11-13T08:11:02.797ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
218712025-11-13T08:11:02.797ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
218722025-11-13T08:11:02.797ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
218732025-11-13T08:11:02.797ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
218742025-11-13T08:11:02.797ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
218752025-11-13T08:11:02.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
218762025-11-13T08:11:02.798ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = DeployNexusRecords step id = Ensure
218772025-11-13T08:11:02.798ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218782025-11-13T08:11:02.798ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218792025-11-13T08:11:02.799ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218802025-11-13T08:11:02.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
218812025-11-13T08:11:02.800ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
218822025-11-13T08:11:02.800ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 156.554188ms iteration = 73 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
218832025-11-13T08:11:02.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
218842025-11-13T08:11:02.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
218852025-11-13T08:11:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
218862025-11-13T08:11:02.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218872025-11-13T08:11:02.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218882025-11-13T08:11:02.804ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
218892025-11-13T08:11:02.804ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
218902025-11-13T08:11:02.805ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
218912025-11-13T08:11:02.805ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 197.762658ms iteration = 13 status = {"generation":1}
218922025-11-13T08:11:02.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
218932025-11-13T08:11:02.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
218942025-11-13T08:11:02.805ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
218952025-11-13T08:11:02.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
218962025-11-13T08:11:02.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
218972025-11-13T08:11:02.806ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
218982025-11-13T08:11:02.806ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 180.328865ms iteration = 73 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
218992025-11-13T08:11:02.809ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
219002025-11-13T08:11:02.809ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
219012025-11-13T08:11:02.809ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
219022025-11-13T08:11:02.809ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 202.182085ms iteration = 13 status = {"generation":2}
219032025-11-13T08:11:02.813ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219042025-11-13T08:11:02.818ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
219052025-11-13T08:11:02.818ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
219062025-11-13T08:11:02.818ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 170.090363ms iteration = 25 status = {"errors":[],"requests_created_ok":[]}
219072025-11-13T08:11:02.821ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 214.105169ms iteration = 25 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
219082025-11-13T08:11:02.838ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
219092025-11-13T08:11:02.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219102025-11-13T08:11:02.842ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219112025-11-13T08:11:02.843ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 194.405762ms iteration = 25 status = {"errors":[],"garbage_collect_requested":[]}
219122025-11-13T08:11:02.844ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219132025-11-13T08:11:02.846ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
219142025-11-13T08:11:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
219152025-11-13T08:11:02.850ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 200.992372ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
219162025-11-13T08:11:02.850ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
219172025-11-13T08:11:02.850ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219182025-11-13T08:11:02.850ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219192025-11-13T08:11:02.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219202025-11-13T08:11:02.855ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
219212025-11-13T08:11:02.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219222025-11-13T08:11:02.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219232025-11-13T08:11:02.860ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219242025-11-13T08:11:02.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219252025-11-13T08:11:02.861ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
219262025-11-13T08:11:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 234.948319ms iteration = 37 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
219272025-11-13T08:11:02.862ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 218.902513ms iteration = 25 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
219282025-11-13T08:11:02.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219292025-11-13T08:11:02.863ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219302025-11-13T08:11:02.865ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
219312025-11-13T08:11:02.865ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 222.205919ms iteration = 25 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
219322025-11-13T08:11:02.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219332025-11-13T08:11:02.866ZINFOnexus (BackgroundTasks): found Silo/DNS/TLS config (changed) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) } file = nexus/src/app/background/tasks/external_endpoints.rs:94
219342025-11-13T08:11:02.867ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 259.857489ms iteration = 13 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
219352025-11-13T08:11:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
219362025-11-13T08:11:02.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219372025-11-13T08:11:02.871ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219382025-11-13T08:11:02.874ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219392025-11-13T08:11:02.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219402025-11-13T08:11:02.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219412025-11-13T08:11:02.877ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
219422025-11-13T08:11:02.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
219432025-11-13T08:11:02.879ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
219442025-11-13T08:11:02.879ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
219452025-11-13T08:11:02.879ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 235.488029ms iteration = 25 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
219462025-11-13T08:11:02.880ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 231.97597ms iteration = 25 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
219472025-11-13T08:11:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
219482025-11-13T08:11:02.881ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 255.273718ms iteration = 40 status = {"physical_disks_added":0}
219492025-11-13T08:11:02.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219502025-11-13T08:11:02.881ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219512025-11-13T08:11:02.883ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219522025-11-13T08:11:02.885ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
219532025-11-13T08:11:02.889ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
219542025-11-13T08:11:02.889ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219552025-11-13T08:11:02.889ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219562025-11-13T08:11:02.892ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219572025-11-13T08:11:02.892ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = SledAgent step id = Ensure
219582025-11-13T08:11:02.893ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219592025-11-13T08:11:02.896ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219602025-11-13T08:11:02.899ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
219612025-11-13T08:11:02.900ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "6945e1b3-5c71-45be-9dc1-37a97bb08c98", "content-length": "78", "date": "Thu, 13 Nov 2025 08:11:02 GMT"} })
219622025-11-13T08:11:02.900ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
219632025-11-13T08:11:02.900ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
219642025-11-13T08:11:02.900ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
219652025-11-13T08:11:02.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
219662025-11-13T08:11:02.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
219672025-11-13T08:11:02.900ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
219682025-11-13T08:11:02.901ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
219692025-11-13T08:11:02.901ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219702025-11-13T08:11:02.901ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219712025-11-13T08:11:02.902ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
219722025-11-13T08:11:02.902ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
219732025-11-13T08:11:02.903ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219742025-11-13T08:11:02.903ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
219752025-11-13T08:11:02.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219762025-11-13T08:11:02.905ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
219772025-11-13T08:11:02.905ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
219782025-11-13T08:11:02.905ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219792025-11-13T08:11:02.905ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
219802025-11-13T08:11:02.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
219812025-11-13T08:11:02.907ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
219822025-11-13T08:11:02.907ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
219832025-11-13T08:11:02.907ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
219842025-11-13T08:11:02.907ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
219852025-11-13T08:11:02.908ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
219862025-11-13T08:11:02.911ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
219872025-11-13T08:11:02.911ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
219882025-11-13T08:11:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
219892025-11-13T08:11:02.911ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219902025-11-13T08:11:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219912025-11-13T08:11:02.911ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd92a84d-c30c-433d-8b1c-86cd61e851b8", "content-length": "81", "date": "Thu, 13 Nov 2025 08:11:02 GMT"} })
219922025-11-13T08:11:02.913ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
219932025-11-13T08:11:02.914ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
219942025-11-13T08:11:02.916ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
219952025-11-13T08:11:02.917ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
219962025-11-13T08:11:02.917ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
219972025-11-13T08:11:02.918ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5102141-e61a-495c-a779-c8932735ff00", "content-length": "26", "date": "Thu, 13 Nov 2025 08:11:02 GMT"} })
219982025-11-13T08:11:02.918ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
219992025-11-13T08:11:02.920ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
220002025-11-13T08:11:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
220012025-11-13T08:11:02.921ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220022025-11-13T08:11:02.921ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220032025-11-13T08:11:02.940ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
220042025-11-13T08:11:02.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
220052025-11-13T08:11:02.955ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220062025-11-13T08:11:02.955ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
220072025-11-13T08:11:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220082025-11-13T08:11:02.956ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "73bbc589-f3bf-4978-8d85-4d3c6f82299d", "date": "Thu, 13 Nov 2025 08:11:02 GMT"} })
220092025-11-13T08:11:02.956ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
220102025-11-13T08:11:02.957ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220112025-11-13T08:11:02.958ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
220122025-11-13T08:11:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220132025-11-13T08:11:02.958ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 309.003405ms iteration = 13 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
220142025-11-13T08:11:02.960ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220152025-11-13T08:11:02.964ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
220162025-11-13T08:11:02.964ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
220172025-11-13T08:11:02.964ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6b0fde1-8f09-4647-b362-8989baf1567a", "content-length": "81", "date": "Thu, 13 Nov 2025 08:11:02 GMT"} })
220182025-11-13T08:11:02.965ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
220192025-11-13T08:11:02.965ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220202025-11-13T08:11:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220212025-11-13T08:11:02.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220222025-11-13T08:11:02.970ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220232025-11-13T08:11:02.970ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220242025-11-13T08:11:02.975ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
220252025-11-13T08:11:02.975ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220262025-11-13T08:11:02.975ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220272025-11-13T08:11:02.980ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
220282025-11-13T08:11:02.980ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
220292025-11-13T08:11:02.980ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f3e4001-2a76-4055-88bd-c42b09ce5309", "content-length": "26", "date": "Thu, 13 Nov 2025 08:11:02 GMT"} })
220302025-11-13T08:11:02.980ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
220312025-11-13T08:11:02.981ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220322025-11-13T08:11:02.981ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220332025-11-13T08:11:02.983ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
220342025-11-13T08:11:02.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220352025-11-13T08:11:02.983ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220362025-11-13T08:11:02.985ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
220372025-11-13T08:11:02.985ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
220382025-11-13T08:11:02.985ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "84c73dba-f508-4e5f-a809-0ba7cfa90322", "content-length": "81", "date": "Thu, 13 Nov 2025 08:11:02 GMT"} })
220392025-11-13T08:11:02.985ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220402025-11-13T08:11:02.988ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 339.632507ms iteration = 25 status = {"disabled":false,"errors":[],"sps":[]}
220412025-11-13T08:11:02.990ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
220422025-11-13T08:11:02.990ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220432025-11-13T08:11:02.990ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220442025-11-13T08:11:02.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220452025-11-13T08:11:02.996ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
220462025-11-13T08:11:02.996ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220472025-11-13T08:11:02.996ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220482025-11-13T08:11:02.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220492025-11-13T08:11:03.000ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220502025-11-13T08:11:03.000ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220512025-11-13T08:11:03.001ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220522025-11-13T08:11:03.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220532025-11-13T08:11:03.008ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
220542025-11-13T08:11:03.041ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
220552025-11-13T08:11:03.043ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220562025-11-13T08:11:03.044ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "ce1e8531-21ca-4ed8-bfd7-2b3c68e260ee", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
220572025-11-13T08:11:03.044ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220582025-11-13T08:11:03.045ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = FirewallRules step id = Ensure
220592025-11-13T08:11:03.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220602025-11-13T08:11:03.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220612025-11-13T08:11:03.046ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220622025-11-13T08:11:03.048ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220632025-11-13T08:11:03.048ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220642025-11-13T08:11:03.049ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
220652025-11-13T08:11:03.049ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220662025-11-13T08:11:03.049ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220672025-11-13T08:11:03.050ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220682025-11-13T08:11:03.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220692025-11-13T08:11:03.055ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220702025-11-13T08:11:03.055ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220712025-11-13T08:11:03.056ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220722025-11-13T08:11:03.058ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220732025-11-13T08:11:03.058ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220742025-11-13T08:11:03.058ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
220752025-11-13T08:11:03.059ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
220762025-11-13T08:11:03.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
220772025-11-13T08:11:03.060ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220782025-11-13T08:11:03.060ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220792025-11-13T08:11:03.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220802025-11-13T08:11:03.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220812025-11-13T08:11:03.064ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220822025-11-13T08:11:03.064ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220832025-11-13T08:11:03.065ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220842025-11-13T08:11:03.065ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
220852025-11-13T08:11:03.066ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
220862025-11-13T08:11:03.066ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220872025-11-13T08:11:03.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220882025-11-13T08:11:03.067ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220892025-11-13T08:11:03.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220902025-11-13T08:11:03.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220912025-11-13T08:11:03.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
220922025-11-13T08:11:03.070ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
220932025-11-13T08:11:03.071ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
220942025-11-13T08:11:03.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
220952025-11-13T08:11:03.072ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220962025-11-13T08:11:03.072ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220972025-11-13T08:11:03.075ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
220982025-11-13T08:11:03.075ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
220992025-11-13T08:11:03.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
221002025-11-13T08:11:03.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221012025-11-13T08:11:03.076ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221022025-11-13T08:11:03.077ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221032025-11-13T08:11:03.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221042025-11-13T08:11:03.079ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221052025-11-13T08:11:03.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221062025-11-13T08:11:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
221072025-11-13T08:11:03.081ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
221082025-11-13T08:11:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221092025-11-13T08:11:03.081ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
221102025-11-13T08:11:03.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221112025-11-13T08:11:03.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221122025-11-13T08:11:03.084ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
221132025-11-13T08:11:03.084ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
221142025-11-13T08:11:03.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221152025-11-13T08:11:03.085ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221162025-11-13T08:11:03.085ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221172025-11-13T08:11:03.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221182025-11-13T08:11:03.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221192025-11-13T08:11:03.088ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221202025-11-13T08:11:03.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221212025-11-13T08:11:03.090ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
221222025-11-13T08:11:03.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
221232025-11-13T08:11:03.090ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221242025-11-13T08:11:03.090ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
221252025-11-13T08:11:03.091ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
221262025-11-13T08:11:03.091ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
221272025-11-13T08:11:03.093ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
221282025-11-13T08:11:03.094ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "12e439ea-f0d0-4eb3-981e-3406142faf8b", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
221292025-11-13T08:11:03.094ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
221302025-11-13T08:11:03.095ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221312025-11-13T08:11:03.096ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "9aae40e1-2a79-4f59-99c7-02e35b5470ed", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
221322025-11-13T08:11:03.096ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
221332025-11-13T08:11:03.097ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
221342025-11-13T08:11:03.097ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221352025-11-13T08:11:03.097ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221362025-11-13T08:11:03.098ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
221372025-11-13T08:11:03.098ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 455.432904ms iteration = 13 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
221382025-11-13T08:11:03.099ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
221392025-11-13T08:11:03.099ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221402025-11-13T08:11:03.099ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 491.88074ms iteration = 15 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
221412025-11-13T08:11:03.099ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221422025-11-13T08:11:03.101ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
221432025-11-13T08:11:03.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
221442025-11-13T08:11:03.101ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221452025-11-13T08:11:03.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
221462025-11-13T08:11:03.101ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221472025-11-13T08:11:03.103ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
221482025-11-13T08:11:03.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
221492025-11-13T08:11:03.104ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221502025-11-13T08:11:03.104ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221512025-11-13T08:11:03.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221522025-11-13T08:11:03.106ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221532025-11-13T08:11:03.106ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221542025-11-13T08:11:03.107ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221552025-11-13T08:11:03.108ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221562025-11-13T08:11:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
221572025-11-13T08:11:03.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
221582025-11-13T08:11:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
221592025-11-13T08:11:03.109ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221602025-11-13T08:11:03.109ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221612025-11-13T08:11:03.111ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221622025-11-13T08:11:03.117ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 510.649429ms iteration = 25 status = {}
221632025-11-13T08:11:03.119ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
221642025-11-13T08:11:03.119ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221652025-11-13T08:11:03.119ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221662025-11-13T08:11:03.121ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221672025-11-13T08:11:03.121ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221682025-11-13T08:11:03.122ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221692025-11-13T08:11:03.122ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221702025-11-13T08:11:03.124ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
221712025-11-13T08:11:03.129ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
221722025-11-13T08:11:03.129ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221732025-11-13T08:11:03.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221742025-11-13T08:11:03.131ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
221752025-11-13T08:11:03.131ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
221762025-11-13T08:11:03.131ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f663d9a-f7a0-455d-983e-76c8802c7c55", "content-length": "372", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
221772025-11-13T08:11:03.131ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
221782025-11-13T08:11:03.131ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
221792025-11-13T08:11:03.133ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6671, tv_nsec: 485631436 } })
221802025-11-13T08:11:03.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221812025-11-13T08:11:03.134ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221822025-11-13T08:11:03.135ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221832025-11-13T08:11:03.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221842025-11-13T08:11:03.138ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221852025-11-13T08:11:03.138ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221862025-11-13T08:11:03.140ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221872025-11-13T08:11:03.140ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221882025-11-13T08:11:03.142ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221892025-11-13T08:11:03.142ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221902025-11-13T08:11:03.144ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221912025-11-13T08:11:03.144ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221922025-11-13T08:11:03.145ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
221932025-11-13T08:11:03.145ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
221942025-11-13T08:11:03.146ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "9d312de4-88e8-44c6-80b5-846c255383be", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
221952025-11-13T08:11:03.146ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
221962025-11-13T08:11:03.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
221972025-11-13T08:11:03.146ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
221982025-11-13T08:11:03.146ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
221992025-11-13T08:11:03.148ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222002025-11-13T08:11:03.148ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
222012025-11-13T08:11:03.148ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
222022025-11-13T08:11:03.148ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
222032025-11-13T08:11:03.148ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
222042025-11-13T08:11:03.149ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = e5915d47-9cf0-4622-b7aa-4c34b38de90c parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
222052025-11-13T08:11:03.149ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
222062025-11-13T08:11:03.149ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
222072025-11-13T08:11:03.151ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222082025-11-13T08:11:03.151ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
222092025-11-13T08:11:03.151ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
222102025-11-13T08:11:03.154ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
222112025-11-13T08:11:03.154ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
222122025-11-13T08:11:03.154ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "f9c30eb3-acb8-41ef-9f8e-890b062ff0e2", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
222132025-11-13T08:11:03.155ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 512.338283ms iteration = 37 status = {}
222142025-11-13T08:11:03.155ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 38 reason = Dependency
222152025-11-13T08:11:03.155ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
222162025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
222172025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
222182025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
222192025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
222202025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
222212025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
222222025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
222232025-11-13T08:11:03.156ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
222242025-11-13T08:11:03.157ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222252025-11-13T08:11:03.158ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
222262025-11-13T08:11:03.158ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
222272025-11-13T08:11:03.159ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
222282025-11-13T08:11:03.159ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
222292025-11-13T08:11:03.159ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
222302025-11-13T08:11:03.159ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
222312025-11-13T08:11:03.159ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
222322025-11-13T08:11:03.159ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 541.856043ms iteration = 30 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
222332025-11-13T08:11:03.160ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
222342025-11-13T08:11:03.160ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
222352025-11-13T08:11:03.161ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222362025-11-13T08:11:03.161ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222372025-11-13T08:11:03.164ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
222382025-11-13T08:11:03.164ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
222392025-11-13T08:11:03.165ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
222402025-11-13T08:11:03.166ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
222412025-11-13T08:11:03.166ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
222422025-11-13T08:11:03.167ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
222432025-11-13T08:11:03.169ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
222442025-11-13T08:11:03.169ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "08fca6f1-052e-4ca0-8adb-98351bbd4f53", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
222452025-11-13T08:11:03.169ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
222462025-11-13T08:11:03.172ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "a71a9d46-c0f4-49d2-bcf3-f9ca4903602f", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
222472025-11-13T08:11:03.172ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
222482025-11-13T08:11:03.173ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
222492025-11-13T08:11:03.173ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
222502025-11-13T08:11:03.173ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
222512025-11-13T08:11:03.174ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "a9489031-0b2b-4cd8-9df5-af8e56c7fe66", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
222522025-11-13T08:11:03.174ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
222532025-11-13T08:11:03.174ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
222542025-11-13T08:11:03.176ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
222552025-11-13T08:11:03.176ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 569.603898ms iteration = 15 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
222562025-11-13T08:11:03.177ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
222572025-11-13T08:11:03.177ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
222582025-11-13T08:11:03.177ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
222592025-11-13T08:11:03.179ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
222602025-11-13T08:11:03.179ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
222612025-11-13T08:11:03.179ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
222622025-11-13T08:11:03.193ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
222632025-11-13T08:11:03.193ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "5333d09c-c73d-4ec2-ab6f-dbc48b8f6544", "content-length": "691", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
222642025-11-13T08:11:03.193ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
222652025-11-13T08:11:03.193ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
222662025-11-13T08:11:03.193ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
222672025-11-13T08:11:03.195ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 551.741827ms iteration = 25 status = {}
222682025-11-13T08:11:03.223ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
222692025-11-13T08:11:03.223ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
222702025-11-13T08:11:03.223ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "c24e79e7-7e99-449f-90f7-237d4c95bd69", "content-length": "372", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
222712025-11-13T08:11:03.223ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
222722025-11-13T08:11:03.223ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
222732025-11-13T08:11:03.242ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
222742025-11-13T08:11:03.243ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
222752025-11-13T08:11:03.245ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
222762025-11-13T08:11:03.245ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
222772025-11-13T08:11:03.245ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
222782025-11-13T08:11:03.253ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
222792025-11-13T08:11:03.253ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
222802025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
222812025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
222822025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
222832025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
222842025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
222852025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
222862025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
222872025-11-13T08:11:03.262ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
222882025-11-13T08:11:03.263ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
222892025-11-13T08:11:03.263ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
222902025-11-13T08:11:03.263ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
222912025-11-13T08:11:03.263ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
222922025-11-13T08:11:03.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
222932025-11-13T08:11:03.266ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
222942025-11-13T08:11:03.266ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
222952025-11-13T08:11:03.287ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
222962025-11-13T08:11:03.287ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
222972025-11-13T08:11:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
222982025-11-13T08:11:03.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
222992025-11-13T08:11:03.291ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223002025-11-13T08:11:03.291ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223012025-11-13T08:11:03.293ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "2e580366-0ab9-4ab5-9ae6-0865ee865787", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
223022025-11-13T08:11:03.293ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
223032025-11-13T08:11:03.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223042025-11-13T08:11:03.293ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
223052025-11-13T08:11:03.295ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
223062025-11-13T08:11:03.295ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
223072025-11-13T08:11:03.296ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
223082025-11-13T08:11:03.296ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
223092025-11-13T08:11:03.297ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "bdd29a4e-dafd-464f-97b6-c305d7cc149b", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
223102025-11-13T08:11:03.298ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
223112025-11-13T08:11:03.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
223122025-11-13T08:11:03.298ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
223132025-11-13T08:11:03.298ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = Dns step id = Ensure
223142025-11-13T08:11:03.298ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
223152025-11-13T08:11:03.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223162025-11-13T08:11:03.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223172025-11-13T08:11:03.299ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223182025-11-13T08:11:03.299ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223192025-11-13T08:11:03.300ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
223202025-11-13T08:11:03.300ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
223212025-11-13T08:11:03.301ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223222025-11-13T08:11:03.301ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "cc625002-0fc0-444d-a7ac-a31f0eeeed37", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
223232025-11-13T08:11:03.303ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 147.266125ms iteration = 38 status = {}
223242025-11-13T08:11:03.304ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
223252025-11-13T08:11:03.305ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
223262025-11-13T08:11:03.305ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
223272025-11-13T08:11:03.305ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
223282025-11-13T08:11:03.305ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223292025-11-13T08:11:03.305ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223302025-11-13T08:11:03.306ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223312025-11-13T08:11:03.310ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
223322025-11-13T08:11:03.310ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223332025-11-13T08:11:03.310ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223342025-11-13T08:11:03.312ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
223352025-11-13T08:11:03.312ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
223362025-11-13T08:11:03.312ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223372025-11-13T08:11:03.313ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223382025-11-13T08:11:03.314ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223392025-11-13T08:11:03.317ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
223402025-11-13T08:11:03.317ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223412025-11-13T08:11:03.317ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223422025-11-13T08:11:03.320ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
223432025-11-13T08:11:03.320ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
223442025-11-13T08:11:03.320ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
223452025-11-13T08:11:03.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223462025-11-13T08:11:03.320ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223472025-11-13T08:11:03.320ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223482025-11-13T08:11:03.321ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223492025-11-13T08:11:03.322ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223502025-11-13T08:11:03.325ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
223512025-11-13T08:11:03.326ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
223522025-11-13T08:11:03.326ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
223532025-11-13T08:11:03.326ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
223542025-11-13T08:11:03.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223552025-11-13T08:11:03.326ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223562025-11-13T08:11:03.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223572025-11-13T08:11:03.331ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
223582025-11-13T08:11:03.331ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223592025-11-13T08:11:03.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223602025-11-13T08:11:03.332ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
223612025-11-13T08:11:03.332ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
223622025-11-13T08:11:03.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223632025-11-13T08:11:03.332ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223642025-11-13T08:11:03.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223652025-11-13T08:11:03.337ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
223662025-11-13T08:11:03.337ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223672025-11-13T08:11:03.337ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223682025-11-13T08:11:03.338ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
223692025-11-13T08:11:03.338ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
223702025-11-13T08:11:03.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223712025-11-13T08:11:03.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223722025-11-13T08:11:03.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223732025-11-13T08:11:03.341ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
223742025-11-13T08:11:03.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223752025-11-13T08:11:03.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223762025-11-13T08:11:03.343ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223772025-11-13T08:11:03.343ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223782025-11-13T08:11:03.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223792025-11-13T08:11:03.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223802025-11-13T08:11:03.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223812025-11-13T08:11:03.351ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
223822025-11-13T08:11:03.353ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
223832025-11-13T08:11:03.353ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
223842025-11-13T08:11:03.353ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = OmicronZones step id = Cleanup
223852025-11-13T08:11:03.353ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = OmicronZones step id = Cleanup
223862025-11-13T08:11:03.353ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = PhysicalDisks step id = Cleanup
223872025-11-13T08:11:03.353ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = Clickhouse step id = Ensure
223882025-11-13T08:11:03.353ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = Clickhouse step id = Ensure
223892025-11-13T08:11:03.383ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
223902025-11-13T08:11:03.388ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "26a30faf-07e7-40a0-a7c6-8075b1fa4b48", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
223912025-11-13T08:11:03.389ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = SupportBundles step id = Cleanup
223922025-11-13T08:11:03.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223932025-11-13T08:11:03.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223942025-11-13T08:11:03.391ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
223952025-11-13T08:11:03.392ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
223962025-11-13T08:11:03.392ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223972025-11-13T08:11:03.392ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
223982025-11-13T08:11:03.393ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
223992025-11-13T08:11:03.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
224002025-11-13T08:11:03.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224012025-11-13T08:11:03.399ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
224022025-11-13T08:11:03.403ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "995307fd-2f7f-42fe-8506-ffacc98b7f32", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224032025-11-13T08:11:03.403ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
224042025-11-13T08:11:03.403ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
224052025-11-13T08:11:03.410ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd7e12d1-40f5-44de-ba8b-41782a3b0fe4", "content-length": "682", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224062025-11-13T08:11:03.410ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
224072025-11-13T08:11:03.411ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
224082025-11-13T08:11:03.414ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
224092025-11-13T08:11:03.414ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
224102025-11-13T08:11:03.417ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
224112025-11-13T08:11:03.417ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
224122025-11-13T08:11:03.419ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
224132025-11-13T08:11:03.419ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
224142025-11-13T08:11:03.426ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 782.37238ms iteration = 25 status = {}
224152025-11-13T08:11:03.447ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = OmicronZones step id = Cleanup
224162025-11-13T08:11:03.447ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
224172025-11-13T08:11:03.447ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
224182025-11-13T08:11:03.447ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
224192025-11-13T08:11:03.447ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
224202025-11-13T08:11:03.450ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224212025-11-13T08:11:03.452ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
224222025-11-13T08:11:03.452ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
224232025-11-13T08:11:03.452ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
224242025-11-13T08:11:03.486ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
224252025-11-13T08:11:03.486ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
224262025-11-13T08:11:03.486ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = Cockroach step id = Ensure
224272025-11-13T08:11:03.486ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
224282025-11-13T08:11:03.487ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
224292025-11-13T08:11:03.500ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
224302025-11-13T08:11:03.500ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = ea2e006f-6535-4a21-a2f4-aee376ebc96b step component = MgsUpdates step id = Ensure
224312025-11-13T08:11:03.500ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Dependency
224322025-11-13T08:11:03.500ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 38 reason = Dependency
224332025-11-13T08:11:03.500ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224342025-11-13T08:11:03.500ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
224352025-11-13T08:11:03.500ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 892.786561ms iteration = 14 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3500,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":146121392,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":146121352,"secs":0}},"event_index":1,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":146144413,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":44234679,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":44234629,"secs":0}},"event_index":2,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":190392043,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":94315344,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":94315294,"secs":0}},"event_index":3,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":284722758,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":152963717,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":152963677,"secs":0}},"event_index":4,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":437707485,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":253094986,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":253094946,"secs":0}},"event_index":5,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":690815542,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":54537261,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":54537231,"secs":0}},"event_index":6,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":745360253,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3620,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":3580,"secs":0}},"event_index":7,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":745375163,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2630,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2590,"secs":0}},"event_index":8,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":745380013,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3190,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3150,"secs":0}},"event_index":9,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":745384673,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2480,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":2451,"secs":0}},"event_index":10,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":745388734,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":35815295,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":35815255,"secs":0}},"event_index":11,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":781205168,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":58256145,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":58256115,"secs":0}},"event_index":12,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":839471533,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":39406384,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":39406334,"secs":0}},"event_index":13,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":878892398,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":13569047,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":13568967,"secs":0}},"event_index":14,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":892473795,"secs":0}},{"data":{"attempt_elapsed":{"nanos":4990,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4950,"secs":0}},"event_index":15,"execution_id":"da5e075c-9b79-4624-9351-4769bc8f401d","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":892488726,"secs":0}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
224362025-11-13T08:11:03.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
224372025-11-13T08:11:03.501ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224382025-11-13T08:11:03.502ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224392025-11-13T08:11:03.503ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224402025-11-13T08:11:03.504ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
224412025-11-13T08:11:03.505ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224422025-11-13T08:11:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224432025-11-13T08:11:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
224442025-11-13T08:11:03.505ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
224452025-11-13T08:11:03.505ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
224462025-11-13T08:11:03.506ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
224472025-11-13T08:11:03.506ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
224482025-11-13T08:11:03.508ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
224492025-11-13T08:11:03.509ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(41569d69-1e42-4dd0-b148-1d962bb1cb70, 0), (f9644618-9762-4080-98b5-efa02367a4b9, 0), (5c9c60a5-45dd-45ca-8f27-717b1b56d226, 0), (b8358fe6-a8f3-4e1d-81e4-53d438ed69da, 0)] collection_id = 41569d69-1e42-4dd0-b148-1d962bb1cb70
224502025-11-13T08:11:03.509ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224512025-11-13T08:11:03.509ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224522025-11-13T08:11:03.510ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
224532025-11-13T08:11:03.510ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
224542025-11-13T08:11:03.510ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
224552025-11-13T08:11:03.512ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
224562025-11-13T08:11:03.512ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 12.002386ms iteration = 38 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
224572025-11-13T08:11:03.628ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 41569d69-1e42-4dd0-b148-1d962bb1cb70 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
224582025-11-13T08:11:03.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224592025-11-13T08:11:03.628ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224602025-11-13T08:11:03.632ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(f9644618-9762-4080-98b5-efa02367a4b9, 0), (5c9c60a5-45dd-45ca-8f27-717b1b56d226, 0), (b8358fe6-a8f3-4e1d-81e4-53d438ed69da, 0), (03b997c5-48a9-4b4f-8987-8a3341afd066, 0)] collection_id = f9644618-9762-4080-98b5-efa02367a4b9
224612025-11-13T08:11:03.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224622025-11-13T08:11:03.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224632025-11-13T08:11:03.751ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = f9644618-9762-4080-98b5-efa02367a4b9 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 12 nclickhouse_keeper_membership = 0 ncockroach_status = 5 ncollections = 1 nconfig_reconcilers = 1 ndatasets = 93 nerrors = 0 nhost_phase1_active_slots = 1 nhost_phase1_flash_hashes = 2 ninternal_dns = 3 nlast_reconciliation_dataset_results = 75 nlast_reconciliation_disk_results = 9 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 28 nmupdate_override_non_boot = 0 nntp_timesync = 1 nnvme_disk_firmware = 11 nomicron_sled_config_datasets = 75 nomicron_sled_config_disks = 9 nomicron_sled_config_zone_nics = 6 nomicron_sled_config_zones = 28 nomicron_sled_configs = 1 nphysical_disks = 11 nrot_pages = 8 nrots = 2 nsled_agents = 1 nsps = 2 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 9
224642025-11-13T08:11:03.751ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224652025-11-13T08:11:03.752ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224662025-11-13T08:11:03.755ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(5c9c60a5-45dd-45ca-8f27-717b1b56d226, 0), (b8358fe6-a8f3-4e1d-81e4-53d438ed69da, 0), (03b997c5-48a9-4b4f-8987-8a3341afd066, 0), (a7967a9a-1e1f-42c0-b592-9f2752cb200d, 0)] collection_id = 5c9c60a5-45dd-45ca-8f27-717b1b56d226
224672025-11-13T08:11:03.755ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224682025-11-13T08:11:03.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224692025-11-13T08:11:03.895ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 31 transaction = inventory_delete_collection
224702025-11-13T08:11:03.952ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
224712025-11-13T08:11:03.952ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = 5c9c60a5-45dd-45ca-8f27-717b1b56d226 file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
224722025-11-13T08:11:03.952ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
224732025-11-13T08:11:03.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
224742025-11-13T08:11:03.956ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(b8358fe6-a8f3-4e1d-81e4-53d438ed69da, 0), (03b997c5-48a9-4b4f-8987-8a3341afd066, 0), (a7967a9a-1e1f-42c0-b592-9f2752cb200d, 0)]
224752025-11-13T08:11:03.956ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6672, tv_nsec: 308663976 } })
224762025-11-13T08:11:03.993ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6672, tv_nsec: 345795056 } })
224772025-11-13T08:11:03.994ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
224782025-11-13T08:11:03.994ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
224792025-11-13T08:11:03.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
224802025-11-13T08:11:03.994ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1eed4cbe-deba-4a80-b074-e5c5f4e70b67", "content-length": "476", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224812025-11-13T08:11:03.994ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
224822025-11-13T08:11:03.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "750976fe-0251-41aa-b475-af67b3b6b5c9", "content-length": "734", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224832025-11-13T08:11:03.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
224842025-11-13T08:11:03.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35a215f9-9f5f-4bcc-b5d5-04a1d42e4953", "content-length": "111", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224852025-11-13T08:11:03.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
224862025-11-13T08:11:03.995ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0156a7e-cdc7-4f5a-b879-8d82c98ff4ee", "content-length": "111", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224872025-11-13T08:11:03.995ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
224882025-11-13T08:11:03.996ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4be621fe-bc8f-43a8-b4f0-20950cc84b53", "content-length": "163", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224892025-11-13T08:11:03.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
224902025-11-13T08:11:03.997ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8166cf6a-af30-4283-9412-eff2c0991698", "content-length": "163", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224912025-11-13T08:11:03.997ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
224922025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d23c842-772e-486f-b529-6692685383d9", "content-length": "166", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224932025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
224942025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e93c93c9-43ff-4776-b3bc-f337de6e6885", "content-length": "166", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224952025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
224962025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "24b0b8e8-1fc2-4ff5-aad0-d2eb8d228b98", "content-length": "702", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224972025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
224982025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "182e5c01-051d-4f48-9943-31bdacf5ac44", "content-length": "727", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
224992025-11-13T08:11:03.998ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
225002025-11-13T08:11:03.999ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23f97e33-baa6-426b-b09e-330074313b44", "content-length": "729", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
225012025-11-13T08:11:03.999ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
225022025-11-13T08:11:03.999ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c326aee-ed88-42b3-85bb-0c0ab70df652", "content-length": "728", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
225032025-11-13T08:11:03.999ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
225042025-11-13T08:11:03.999ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac8f23ef-c1b1-4fbe-ad9c-c2272e5deceb", "content-length": "722", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
225052025-11-13T08:11:03.999ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
225062025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fddca7e-0e9d-4c84-828d-6ed17f484eca", "content-length": "10", "date": "Thu, 13 Nov 2025 08:11:03 GMT"} })
225072025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
225082025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4246d3a1-9cc0-4a93-bb9c-ddd8342cfe13", "content-length": "144", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225092025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
225102025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "993bddd0-e873-44db-a5ac-030193a7190e", "content-length": "147", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225112025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
225122025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85f7cbc6-6670-48aa-b4fc-d5e7fcdee5bb", "content-length": "109", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225132025-11-13T08:11:04.000ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
225142025-11-13T08:11:04.001ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e232b23-b834-4395-8641-9182c5c52c8b", "content-length": "109", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225152025-11-13T08:11:04.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
225162025-11-13T08:11:04.001ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5486a384-b6c0-4ef9-9f65-571b5ee51896", "content-length": "163", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225172025-11-13T08:11:04.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
225182025-11-13T08:11:04.001ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "149604ff-f37c-48c4-9759-4350b3edddf8", "content-length": "163", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225192025-11-13T08:11:04.001ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
225202025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "839ee1a3-4b8a-4e83-9cdd-73c00415ef68", "content-length": "166", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225212025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
225222025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "185b69d9-d41e-4150-83df-4cd337440292", "content-length": "166", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225232025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
225242025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5343fcd2-c8a3-47c9-8ec4-7b4e02b749fd", "content-length": "702", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225252025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
225262025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e166b41-7119-4379-9f98-9d1965feae18", "content-length": "727", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225272025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
225282025-11-13T08:11:04.002ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d4be389-6477-4916-b6f1-96759dae513d", "content-length": "729", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225292025-11-13T08:11:04.003ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
225302025-11-13T08:11:04.003ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5678987-d36b-4032-aadd-b91fc628fedd", "content-length": "728", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225312025-11-13T08:11:04.003ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
225322025-11-13T08:11:04.003ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
225332025-11-13T08:11:04.005ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225342025-11-13T08:11:04.008ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
225352025-11-13T08:11:04.008ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
225362025-11-13T08:11:04.008ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
225372025-11-13T08:11:04.009ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225382025-11-13T08:11:04.011ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
225392025-11-13T08:11:04.011ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
225402025-11-13T08:11:04.011ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
225412025-11-13T08:11:04.047ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
225422025-11-13T08:11:04.047ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
225432025-11-13T08:11:04.127ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1ce1dc9-ae6f-4fdc-bfdb-4d5ebe401ae8", "content-length": "95408", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225442025-11-13T08:11:04.130ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
225452025-11-13T08:11:04.130ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
225462025-11-13T08:11:04.130ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
225472025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
225482025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
225492025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
225502025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
225512025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
225522025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c4d4256-3329-4b44-b55d-618b53df4798", "content-length": "64", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225532025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
225542025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "f67b037c-4ed6-4d24-8ccf-9917891351ce", "content-length": "64", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225552025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdc6d96f-8545-4a5a-bdb9-e25269677377", "content-length": "64", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225562025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
225572025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd0564d9-78b5-4acb-8978-3ef797f3f7b6", "content-length": "64", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225582025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e14b723-983c-45e4-af0c-67499cd797b1", "content-length": "64", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225592025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
225602025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
225612025-11-13T08:11:04.130ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
225622025-11-13T08:11:04.167ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c822813-30c9-40c8-877d-6f730b5820ae", "content-length": "430257", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225632025-11-13T08:11:04.169ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e4a2b0e-bb67-4104-9cdd-7ff9fab09aa8", "content-length": "415543", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225642025-11-13T08:11:04.169ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ca45159-5bb3-4a5b-926b-727b8dd7a04d", "content-length": "410123", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225652025-11-13T08:11:04.169ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "63743571-6bcb-4cc0-a907-04ecfa2e32ac", "content-length": "409474", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225662025-11-13T08:11:04.184ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "333eb4c2-d45b-48b8-a54c-4afe016015bb", "content-length": "409771", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225672025-11-13T08:11:04.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
225682025-11-13T08:11:04.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
225692025-11-13T08:11:04.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
225702025-11-13T08:11:04.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
225712025-11-13T08:11:04.186ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
225722025-11-13T08:11:04.216ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
225732025-11-13T08:11:04.216ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
225742025-11-13T08:11:04.219ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "e711f198-d0e4-4713-8204-66d7f975e706", "content-length": "121", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225752025-11-13T08:11:04.220ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
225762025-11-13T08:11:04.310ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
225772025-11-13T08:11:04.310ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
225782025-11-13T08:11:04.310ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9e70a15-db11-44bc-abe7-26ae8265ccde", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225792025-11-13T08:11:04.310ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
225802025-11-13T08:11:04.313ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
225812025-11-13T08:11:04.313ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
225822025-11-13T08:11:04.313ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cb96011-8644-449a-ba38-d5e27476e8ee", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225832025-11-13T08:11:04.313ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
225842025-11-13T08:11:04.316ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
225852025-11-13T08:11:04.316ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
225862025-11-13T08:11:04.316ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e09638f-f627-4a7d-a56b-0b666907b5c8", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:11:04 GMT"} })
225872025-11-13T08:11:04.316ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
225882025-11-13T08:11:04.317ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
225892025-11-13T08:11:04.317ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
225902025-11-13T08:11:04.319ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
225912025-11-13T08:11:04.319ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
225922025-11-13T08:11:04.321ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
225932025-11-13T08:11:04.323ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
225942025-11-13T08:11:04.323ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
225952025-11-13T08:11:04.323ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
225962025-11-13T08:11:04.534ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
225972025-11-13T08:11:04.534ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3 time_started = 2025-11-13 08:11:03.994068 UTC
225982025-11-13T08:11:04.534ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 64 reason = Dependency
225992025-11-13T08:11:04.534ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
226002025-11-13T08:11:04.534ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.033650081s iteration = 15 status = {"collection_id":"46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3","time_done":"2025-11-13 08:11:04.317650 UTC","time_started":"2025-11-13 08:11:03.994068 UTC"}
226012025-11-13T08:11:04.534ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
226022025-11-13T08:11:04.536ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226032025-11-13T08:11:04.539ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
226042025-11-13T08:11:04.539ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
226052025-11-13T08:11:04.539ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
226062025-11-13T08:11:04.542ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
226072025-11-13T08:11:04.542ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
226082025-11-13T08:11:04.594ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 19 reason = Dependency
226092025-11-13T08:11:04.594ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 60.485699ms iteration = 64 status = {"Loaded":{"collection_id":"46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3","time_loaded":"2025-11-13T08:11:04.534156187Z","time_started":"2025-11-13T08:11:03.994068Z"}}
226102025-11-13T08:11:04.594ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 31 reason = Dependency
226112025-11-13T08:11:04.594ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 41 reason = Dependency
226122025-11-13T08:11:04.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226132025-11-13T08:11:04.594ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226142025-11-13T08:11:04.594ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
226152025-11-13T08:11:04.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226162025-11-13T08:11:04.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226172025-11-13T08:11:04.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
226182025-11-13T08:11:04.596ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226192025-11-13T08:11:04.596ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226202025-11-13T08:11:04.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226212025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
226222025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
226232025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
226242025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226252025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
226262025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
226272025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226282025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226292025-11-13T08:11:04.600ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226302025-11-13T08:11:04.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226312025-11-13T08:11:04.602ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226322025-11-13T08:11:04.602ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
226332025-11-13T08:11:04.602ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
226342025-11-13T08:11:04.604ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226352025-11-13T08:11:04.604ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
226362025-11-13T08:11:04.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226372025-11-13T08:11:04.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226382025-11-13T08:11:04.606ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
226392025-11-13T08:11:04.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226402025-11-13T08:11:04.606ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226412025-11-13T08:11:04.606ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226422025-11-13T08:11:04.607ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
226432025-11-13T08:11:04.607ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
226442025-11-13T08:11:04.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226452025-11-13T08:11:04.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
226462025-11-13T08:11:04.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226472025-11-13T08:11:04.611ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
226482025-11-13T08:11:04.611ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226492025-11-13T08:11:04.611ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226502025-11-13T08:11:04.612ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226512025-11-13T08:11:04.614ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
226522025-11-13T08:11:04.614ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226532025-11-13T08:11:04.615ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226542025-11-13T08:11:04.616ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
226552025-11-13T08:11:04.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226562025-11-13T08:11:04.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226572025-11-13T08:11:04.618ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 23.545833ms iteration = 41 status = {"physical_disks_added":0}
226582025-11-13T08:11:04.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226592025-11-13T08:11:04.620ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
226602025-11-13T08:11:04.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226612025-11-13T08:11:04.620ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226622025-11-13T08:11:04.622ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226632025-11-13T08:11:04.624ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
226642025-11-13T08:11:04.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
226652025-11-13T08:11:04.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
226662025-11-13T08:11:04.626ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
226672025-11-13T08:11:04.626ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 32.084811ms iteration = 19 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
226682025-11-13T08:11:04.684ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226692025-11-13T08:11:04.684ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226702025-11-13T08:11:04.686ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226712025-11-13T08:11:04.688ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
226722025-11-13T08:11:04.688ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226732025-11-13T08:11:04.688ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226742025-11-13T08:11:04.690ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226752025-11-13T08:11:04.692ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
226762025-11-13T08:11:04.692ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226772025-11-13T08:11:04.692ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226782025-11-13T08:11:04.694ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226792025-11-13T08:11:04.695ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226802025-11-13T08:11:04.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226812025-11-13T08:11:04.696ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226822025-11-13T08:11:04.698ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226832025-11-13T08:11:04.701ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
226842025-11-13T08:11:04.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226852025-11-13T08:11:04.701ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226862025-11-13T08:11:04.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226872025-11-13T08:11:04.703ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226882025-11-13T08:11:04.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226892025-11-13T08:11:04.708ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
226902025-11-13T08:11:04.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226912025-11-13T08:11:04.708ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226922025-11-13T08:11:04.711ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226932025-11-13T08:11:04.714ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
226942025-11-13T08:11:04.714ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226952025-11-13T08:11:04.714ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226962025-11-13T08:11:04.716ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
226972025-11-13T08:11:04.716ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
226982025-11-13T08:11:04.718ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
226992025-11-13T08:11:04.721ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
227002025-11-13T08:11:04.721ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227012025-11-13T08:11:04.721ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227022025-11-13T08:11:04.723ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227032025-11-13T08:11:04.723ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227042025-11-13T08:11:04.725ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227052025-11-13T08:11:04.728ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
227062025-11-13T08:11:04.728ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227072025-11-13T08:11:04.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227082025-11-13T08:11:04.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227092025-11-13T08:11:04.730ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227102025-11-13T08:11:04.731ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227112025-11-13T08:11:04.734ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
227122025-11-13T08:11:04.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227132025-11-13T08:11:04.734ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227142025-11-13T08:11:04.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227152025-11-13T08:11:04.751ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227162025-11-13T08:11:04.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227172025-11-13T08:11:04.753ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227182025-11-13T08:11:04.754ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227192025-11-13T08:11:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227202025-11-13T08:11:04.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227212025-11-13T08:11:04.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227222025-11-13T08:11:04.758ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227232025-11-13T08:11:04.761ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227242025-11-13T08:11:04.761ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227252025-11-13T08:11:04.761ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227262025-11-13T08:11:04.763ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227272025-11-13T08:11:04.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227282025-11-13T08:11:04.765ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227292025-11-13T08:11:04.765ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227302025-11-13T08:11:04.766ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227312025-11-13T08:11:04.769ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227322025-11-13T08:11:04.769ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227332025-11-13T08:11:04.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227342025-11-13T08:11:04.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227352025-11-13T08:11:04.774ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
227362025-11-13T08:11:04.774ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227372025-11-13T08:11:04.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227382025-11-13T08:11:04.776ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227392025-11-13T08:11:04.776ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227402025-11-13T08:11:04.777ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227412025-11-13T08:11:04.777ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227422025-11-13T08:11:04.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227432025-11-13T08:11:04.781ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
227442025-11-13T08:11:04.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227452025-11-13T08:11:04.782ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227462025-11-13T08:11:04.782ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227472025-11-13T08:11:04.783ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227482025-11-13T08:11:04.784ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227492025-11-13T08:11:04.787ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
227502025-11-13T08:11:04.788ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227512025-11-13T08:11:04.788ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227522025-11-13T08:11:04.790ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227532025-11-13T08:11:04.790ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227542025-11-13T08:11:04.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227552025-11-13T08:11:04.794ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
227562025-11-13T08:11:04.794ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227572025-11-13T08:11:04.794ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227582025-11-13T08:11:04.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227592025-11-13T08:11:04.796ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227602025-11-13T08:11:04.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227612025-11-13T08:11:04.800ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
227622025-11-13T08:11:04.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227632025-11-13T08:11:04.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227642025-11-13T08:11:04.801ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227652025-11-13T08:11:04.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227662025-11-13T08:11:04.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227672025-11-13T08:11:04.805ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
227682025-11-13T08:11:04.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227692025-11-13T08:11:04.805ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227702025-11-13T08:11:04.807ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227712025-11-13T08:11:04.807ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227722025-11-13T08:11:04.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227732025-11-13T08:11:04.810ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
227742025-11-13T08:11:04.810ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227752025-11-13T08:11:04.810ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227762025-11-13T08:11:04.847ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
227772025-11-13T08:11:04.847ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
227782025-11-13T08:11:04.847ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = f81783be-caac-4afd-b668-c2555c28389c parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
227792025-11-13T08:11:04.847ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227802025-11-13T08:11:04.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227812025-11-13T08:11:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227822025-11-13T08:11:04.851ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
227832025-11-13T08:11:04.851ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
227842025-11-13T08:11:04.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
227852025-11-13T08:11:04.854ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
227862025-11-13T08:11:04.854ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
227872025-11-13T08:11:04.854ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 260.106914ms iteration = 31 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
227882025-11-13T08:11:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 447 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = e759a87f-844f-4d88-9515-684dfe055085 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
227892025-11-13T08:11:07.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 146 reason = Timeout
227902025-11-13T08:11:07.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
227912025-11-13T08:11:07.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
227922025-11-13T08:11:07.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
227932025-11-13T08:11:07.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
227942025-11-13T08:11:07.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
227952025-11-13T08:11:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
227962025-11-13T08:11:07.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
227972025-11-13T08:11:07.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.929846ms iteration = 146 status = {"config_updated":false}
227982025-11-13T08:11:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 74 reason = Timeout
227992025-11-13T08:11:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 147 reason = Timeout
228002025-11-13T08:11:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228012025-11-13T08:11:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228022025-11-13T08:11:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228032025-11-13T08:11:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228042025-11-13T08:11:12.627ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228052025-11-13T08:11:12.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228062025-11-13T08:11:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
228072025-11-13T08:11:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
228082025-11-13T08:11:12.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228092025-11-13T08:11:12.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228102025-11-13T08:11:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228112025-11-13T08:11:12.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228122025-11-13T08:11:12.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
228132025-11-13T08:11:12.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.289813ms iteration = 147 status = {"config_updated":false}
228142025-11-13T08:11:12.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228152025-11-13T08:11:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228162025-11-13T08:11:12.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228172025-11-13T08:11:12.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
228182025-11-13T08:11:12.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228192025-11-13T08:11:12.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228202025-11-13T08:11:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 74 reason = Timeout
228212025-11-13T08:11:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
228222025-11-13T08:11:12.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
228232025-11-13T08:11:12.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
228242025-11-13T08:11:12.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
228252025-11-13T08:11:12.649ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
228262025-11-13T08:11:12.649ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 23.461002ms iteration = 74 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
228272025-11-13T08:11:12.679ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.256092ms iteration = 74 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
228282025-11-13T08:11:15.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 322 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 7ff7cc58-d236-48c7-8827-5d2f73347e28 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
228292025-11-13T08:11:17.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 65 reason = Timeout
228302025-11-13T08:11:17.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 148 reason = Timeout
228312025-11-13T08:11:17.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
228322025-11-13T08:11:17.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228332025-11-13T08:11:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228342025-11-13T08:11:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
228352025-11-13T08:11:17.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228362025-11-13T08:11:17.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228372025-11-13T08:11:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
228382025-11-13T08:11:17.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
228392025-11-13T08:11:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
228402025-11-13T08:11:17.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228412025-11-13T08:11:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
228422025-11-13T08:11:17.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228432025-11-13T08:11:17.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
228442025-11-13T08:11:17.633ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3 original_time_started = 2025-11-13 08:11:03.994068 UTC
228452025-11-13T08:11:17.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.178511ms iteration = 148 status = {"config_updated":false}
228462025-11-13T08:11:17.633ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 7.201351ms iteration = 65 status = {"Loaded":{"collection_id":"46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3","time_loaded":"2025-11-13T08:11:17.626417275Z","time_started":"2025-11-13T08:11:03.994068Z"}}
228472025-11-13T08:11:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 50 reason = Timeout
228482025-11-13T08:11:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
228492025-11-13T08:11:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 50 reason = Timeout
228502025-11-13T08:11:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
228512025-11-13T08:11:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
228522025-11-13T08:11:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
228532025-11-13T08:11:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
228542025-11-13T08:11:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
228552025-11-13T08:11:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228562025-11-13T08:11:17.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228572025-11-13T08:11:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
228582025-11-13T08:11:17.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
228592025-11-13T08:11:17.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
228602025-11-13T08:11:17.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
228612025-11-13T08:11:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 6.224312ms iteration = 50 status = "NoSitrep"
228622025-11-13T08:11:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
228632025-11-13T08:11:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
228642025-11-13T08:11:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
228652025-11-13T08:11:17.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
228662025-11-13T08:11:17.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228672025-11-13T08:11:17.659ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
228682025-11-13T08:11:17.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
228692025-11-13T08:11:17.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 11.17965ms iteration = 50 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
228702025-11-13T08:11:22.625ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 75 reason = Timeout
228712025-11-13T08:11:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 149 reason = Timeout
228722025-11-13T08:11:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228732025-11-13T08:11:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228742025-11-13T08:11:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228752025-11-13T08:11:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228762025-11-13T08:11:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228772025-11-13T08:11:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228782025-11-13T08:11:22.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
228792025-11-13T08:11:22.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
228802025-11-13T08:11:22.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
228812025-11-13T08:11:22.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228822025-11-13T08:11:22.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
228832025-11-13T08:11:22.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228842025-11-13T08:11:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228852025-11-13T08:11:22.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
228862025-11-13T08:11:22.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.062128ms iteration = 149 status = {"config_updated":false}
228872025-11-13T08:11:22.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228882025-11-13T08:11:22.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
228892025-11-13T08:11:22.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
228902025-11-13T08:11:22.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
228912025-11-13T08:11:22.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
228922025-11-13T08:11:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 75 reason = Timeout
228932025-11-13T08:11:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
228942025-11-13T08:11:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
228952025-11-13T08:11:22.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
228962025-11-13T08:11:22.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
228972025-11-13T08:11:22.644ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.767765ms iteration = 75 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
228982025-11-13T08:11:22.648ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
228992025-11-13T08:11:22.648ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 22.758857ms iteration = 75 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
229002025-11-13T08:11:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 452 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 01727d9c-74e9-485e-9a16-7749902389c1 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
229012025-11-13T08:11:27.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 150 reason = Timeout
229022025-11-13T08:11:27.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229032025-11-13T08:11:27.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229042025-11-13T08:11:27.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229052025-11-13T08:11:27.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
229062025-11-13T08:11:27.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229072025-11-13T08:11:27.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229082025-11-13T08:11:27.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
229092025-11-13T08:11:27.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.603409ms iteration = 150 status = {"config_updated":false}
229102025-11-13T08:11:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 26 reason = Timeout
229112025-11-13T08:11:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 26 reason = Timeout
229122025-11-13T08:11:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
229132025-11-13T08:11:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 26 reason = Timeout
229142025-11-13T08:11:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
229152025-11-13T08:11:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
229162025-11-13T08:11:32.607ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
229172025-11-13T08:11:32.610ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6700, tv_nsec: 962421219 } })
229182025-11-13T08:11:32.610ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6700, tv_nsec: 962608453 } })
229192025-11-13T08:11:32.610ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
229202025-11-13T08:11:32.611ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 4.12001ms iteration = 26 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
229212025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 39 reason = Timeout
229222025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
229232025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 66 reason = Timeout
229242025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 42 reason = Timeout
229252025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
229262025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
229272025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 76 reason = Timeout
229282025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229292025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 151 reason = Timeout
229302025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229312025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229322025-11-13T08:11:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
229332025-11-13T08:11:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
229342025-11-13T08:11:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
229352025-11-13T08:11:32.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229362025-11-13T08:11:32.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229372025-11-13T08:11:32.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229382025-11-13T08:11:32.630ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229392025-11-13T08:11:32.630ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229402025-11-13T08:11:32.631ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229412025-11-13T08:11:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
229422025-11-13T08:11:32.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
229432025-11-13T08:11:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
229442025-11-13T08:11:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
229452025-11-13T08:11:32.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229462025-11-13T08:11:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
229472025-11-13T08:11:32.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
229482025-11-13T08:11:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229492025-11-13T08:11:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
229502025-11-13T08:11:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
229512025-11-13T08:11:32.636ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
229522025-11-13T08:11:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
229532025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
229542025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
229552025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
229562025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
229572025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
229582025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
229592025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
229602025-11-13T08:11:32.637ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
229612025-11-13T08:11:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
229622025-11-13T08:11:32.638ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
229632025-11-13T08:11:32.638ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 11.890684ms iteration = 151 status = {"config_updated":false}
229642025-11-13T08:11:32.639ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229652025-11-13T08:11:32.639ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229662025-11-13T08:11:32.640ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
229672025-11-13T08:11:32.640ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3 original_time_started = 2025-11-13 08:11:03.994068 UTC
229682025-11-13T08:11:32.640ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 13.914664ms iteration = 66 status = {"Loaded":{"collection_id":"46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3","time_loaded":"2025-11-13T08:11:32.626384433Z","time_started":"2025-11-13T08:11:03.994068Z"}}
229692025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 26 reason = Timeout
229702025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 26 reason = Timeout
229712025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
229722025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
229732025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 26 reason = Timeout
229742025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
229752025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 26 reason = Timeout
229762025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
229772025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 26 reason = Timeout
229782025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
229792025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 76 reason = Timeout
229802025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
229812025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 26 reason = Timeout
229822025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
229832025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
229842025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 39 reason = Timeout
229852025-11-13T08:11:32.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
229862025-11-13T08:11:32.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
229872025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
229882025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
229892025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
229902025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
229912025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
229922025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
229932025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
229942025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
229952025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
229962025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
229972025-11-13T08:11:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
229982025-11-13T08:11:32.644ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
229992025-11-13T08:11:32.644ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
230002025-11-13T08:11:32.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
230012025-11-13T08:11:32.645ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230022025-11-13T08:11:32.645ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230032025-11-13T08:11:32.645ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
230042025-11-13T08:11:32.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
230052025-11-13T08:11:32.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
230062025-11-13T08:11:32.646ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230072025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 26 reason = Timeout
230082025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
230092025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 26 reason = Timeout
230102025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 26 reason = Timeout
230112025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
230122025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
230132025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 51 reason = Timeout
230142025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
230152025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 26 reason = Timeout
230162025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 26 reason = Timeout
230172025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
230182025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 51 reason = Timeout
230192025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
230202025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 26 reason = Timeout
230212025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
230222025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
230232025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
230242025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
230252025-11-13T08:11:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
230262025-11-13T08:11:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
230272025-11-13T08:11:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
230282025-11-13T08:11:32.649ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
230292025-11-13T08:11:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
230302025-11-13T08:11:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6701, tv_nsec: 1353045 } })
230312025-11-13T08:11:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
230322025-11-13T08:11:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
230332025-11-13T08:11:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
230342025-11-13T08:11:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
230352025-11-13T08:11:32.650ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
230362025-11-13T08:11:32.650ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 2.301965ms iteration = 26 status = {"errors":[],"finish_invoked_ok":[]}
230372025-11-13T08:11:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
230382025-11-13T08:11:32.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230392025-11-13T08:11:32.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
230402025-11-13T08:11:32.651ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
230412025-11-13T08:11:32.651ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
230422025-11-13T08:11:32.651ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230432025-11-13T08:11:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
230442025-11-13T08:11:32.652ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
230452025-11-13T08:11:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
230462025-11-13T08:11:32.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230472025-11-13T08:11:32.653ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230482025-11-13T08:11:32.655ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
230492025-11-13T08:11:32.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
230502025-11-13T08:11:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
230512025-11-13T08:11:32.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
230522025-11-13T08:11:32.657ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6701, tv_nsec: 9911903 } })
230532025-11-13T08:11:32.657ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
230542025-11-13T08:11:32.657ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
230552025-11-13T08:11:32.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
230562025-11-13T08:11:32.658ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
230572025-11-13T08:11:32.658ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 9.899305ms iteration = 51 status = "NoSitrep"
230582025-11-13T08:11:32.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
230592025-11-13T08:11:32.658ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
230602025-11-13T08:11:32.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
230612025-11-13T08:11:32.660ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
230622025-11-13T08:11:32.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
230632025-11-13T08:11:32.661ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
230642025-11-13T08:11:32.661ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
230652025-11-13T08:11:32.661ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
230662025-11-13T08:11:32.661ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 19.246339ms iteration = 26 status = 0
230672025-11-13T08:11:32.662ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
230682025-11-13T08:11:32.662ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
230692025-11-13T08:11:32.663ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 14.626097ms iteration = 26 status = {"errors":[],"requests_created_ok":[]}
230702025-11-13T08:11:32.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
230712025-11-13T08:11:32.663ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 15.393793ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
230722025-11-13T08:11:32.664ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
230732025-11-13T08:11:32.667ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
230742025-11-13T08:11:32.668ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
230752025-11-13T08:11:32.668ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 19.814219ms iteration = 51 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
230762025-11-13T08:11:32.685ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
230772025-11-13T08:11:32.685ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 58.650143ms iteration = 76 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
230782025-11-13T08:11:32.708ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
230792025-11-13T08:11:32.708ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
230802025-11-13T08:11:32.708ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "b702785b-22b2-411e-bda2-66001aba6de7", "content-length": "26", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
230812025-11-13T08:11:32.708ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
230822025-11-13T08:11:32.709ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
230832025-11-13T08:11:32.709ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
230842025-11-13T08:11:32.710ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
230852025-11-13T08:11:32.710ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
230862025-11-13T08:11:32.710ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
230872025-11-13T08:11:32.710ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
230882025-11-13T08:11:32.710ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
230892025-11-13T08:11:32.711ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
230902025-11-13T08:11:32.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
230912025-11-13T08:11:32.711ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
230922025-11-13T08:11:32.711ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
230932025-11-13T08:11:32.713ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 70.626598ms iteration = 26 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
230942025-11-13T08:11:32.713ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
230952025-11-13T08:11:32.713ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 87.211434ms iteration = 39 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
230962025-11-13T08:11:32.714ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 65.730322ms iteration = 26 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
230972025-11-13T08:11:32.714ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 65.768204ms iteration = 26 status = {"errors":[],"garbage_collect_requested":[]}
230982025-11-13T08:11:32.714ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
230992025-11-13T08:11:32.715ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
231002025-11-13T08:11:32.721ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
231012025-11-13T08:11:32.723ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 97.395495ms iteration = 42 status = {"physical_disks_added":0}
231022025-11-13T08:11:32.723ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 116.718174ms iteration = 26 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
231032025-11-13T08:11:32.725ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
231042025-11-13T08:11:32.725ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 82.513003ms iteration = 26 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
231052025-11-13T08:11:32.727ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
231062025-11-13T08:11:32.727ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
231072025-11-13T08:11:32.729ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231082025-11-13T08:11:32.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
231092025-11-13T08:11:32.732ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
231102025-11-13T08:11:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
231112025-11-13T08:11:32.734ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
231122025-11-13T08:11:32.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
231132025-11-13T08:11:32.736ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
231142025-11-13T08:11:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
231152025-11-13T08:11:32.761ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
231162025-11-13T08:11:32.761ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
231172025-11-13T08:11:32.761ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ce8bfb6-69d1-436b-b656-9b59c8f5d819", "content-length": "26", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231182025-11-13T08:11:32.761ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
231192025-11-13T08:11:32.762ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
231202025-11-13T08:11:32.762ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c8123bc-0893-45d0-97db-b2e850ba793b", "content-length": "78", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231212025-11-13T08:11:32.762ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
231222025-11-13T08:11:32.762ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
231232025-11-13T08:11:32.762ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
231242025-11-13T08:11:32.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
231252025-11-13T08:11:32.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
231262025-11-13T08:11:32.763ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
231272025-11-13T08:11:32.763ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
231282025-11-13T08:11:32.764ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 121.530499ms iteration = 76 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
231292025-11-13T08:11:32.765ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
231302025-11-13T08:11:32.765ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
231312025-11-13T08:11:32.765ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
231322025-11-13T08:11:32.765ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
231332025-11-13T08:11:32.765ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
231342025-11-13T08:11:32.765ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
231352025-11-13T08:11:32.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
231362025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
231372025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
231382025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
231392025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
231402025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
231412025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
231422025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
231432025-11-13T08:11:32.768ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
231442025-11-13T08:11:32.769ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
231452025-11-13T08:11:32.770ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
231462025-11-13T08:11:32.770ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
231472025-11-13T08:11:32.770ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "5708f4c5-3d9b-4c60-9a01-0ffbc2f34648", "content-length": "81", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231482025-11-13T08:11:32.797ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 190.392483ms iteration = 26 status = {}
231492025-11-13T08:11:32.803ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
231502025-11-13T08:11:32.803ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
231512025-11-13T08:11:32.803ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 160.900213ms iteration = 26 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
231522025-11-13T08:11:32.805ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
231532025-11-13T08:11:32.805ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
231542025-11-13T08:11:32.805ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "638be058-b23d-4dca-8392-7aaf971a93e1", "content-length": "81", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231552025-11-13T08:11:32.811ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
231562025-11-13T08:11:32.811ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
231572025-11-13T08:11:32.812ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "93f0c05d-21a5-4f89-9d29-f212eb30add5", "content-length": "81", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231582025-11-13T08:11:32.814ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 165.991374ms iteration = 26 status = {"disabled":false,"errors":[],"sps":[]}
231592025-11-13T08:11:32.841ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
231602025-11-13T08:11:32.841ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "be15d4ca-db3d-4368-941e-46900ef65f54", "content-length": "691", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231612025-11-13T08:11:32.841ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
231622025-11-13T08:11:32.841ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
231632025-11-13T08:11:32.841ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
231642025-11-13T08:11:32.842ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 200.013982ms iteration = 26 status = {}
231652025-11-13T08:11:32.851ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6701, tv_nsec: 203803915 } })
231662025-11-13T08:11:32.855ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
231672025-11-13T08:11:32.855ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
231682025-11-13T08:11:32.855ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "a030be5b-c78f-4334-bf3a-2321435afe29", "content-length": "372", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231692025-11-13T08:11:32.855ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
231702025-11-13T08:11:32.856ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
231712025-11-13T08:11:32.873ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
231722025-11-13T08:11:32.873ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
231732025-11-13T08:11:32.873ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "4436ffc6-87ee-445c-be64-0e3ed0d071fe", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231742025-11-13T08:11:32.873ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
231752025-11-13T08:11:32.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
231762025-11-13T08:11:32.876ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
231772025-11-13T08:11:32.876ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
231782025-11-13T08:11:32.879ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
231792025-11-13T08:11:32.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
231802025-11-13T08:11:32.883ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
231812025-11-13T08:11:32.883ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
231822025-11-13T08:11:32.884ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "4a010daf-a954-424f-84cd-6c2ce163fb18", "date": "Thu, 13 Nov 2025 08:11:32 GMT"} })
231832025-11-13T08:11:32.886ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 243.536908ms iteration = 39 status = {}
231842025-11-13T08:11:32.917ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
231852025-11-13T08:11:32.918ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
231862025-11-13T08:11:32.920ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
231872025-11-13T08:11:32.920ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
231882025-11-13T08:11:32.920ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
231892025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
231902025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
231912025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
231922025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
231932025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
231942025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
231952025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
231962025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
231972025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
231982025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
231992025-11-13T08:11:32.932ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
232002025-11-13T08:11:32.932ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232012025-11-13T08:11:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232022025-11-13T08:11:32.935ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
232032025-11-13T08:11:32.935ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
232042025-11-13T08:11:33.193ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "93e90ba0-a874-4a8c-99b6-3197d8a185bd", "date": "Thu, 13 Nov 2025 08:11:33 GMT"} })
232052025-11-13T08:11:33.194ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
232062025-11-13T08:11:33.194ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
232072025-11-13T08:11:33.194ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b86ad834-dbf4-448e-9d61-a4c753e3d7f1", "content-length": "682", "date": "Thu, 13 Nov 2025 08:11:33 GMT"} })
232082025-11-13T08:11:33.194ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232092025-11-13T08:11:33.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232102025-11-13T08:11:33.198ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232112025-11-13T08:11:33.198ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232122025-11-13T08:11:33.202ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232132025-11-13T08:11:33.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232142025-11-13T08:11:33.204ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
232152025-11-13T08:11:33.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
232162025-11-13T08:11:33.211ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 568.519747ms iteration = 26 status = {}
232172025-11-13T08:11:35.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 413 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = c5a84079-b14f-4b5e-99ca-1241263eb9b4 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
232182025-11-13T08:11:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 152 reason = Timeout
232192025-11-13T08:11:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
232202025-11-13T08:11:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
232212025-11-13T08:11:37.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232222025-11-13T08:11:37.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
232232025-11-13T08:11:37.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
232242025-11-13T08:11:37.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
232252025-11-13T08:11:37.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
232262025-11-13T08:11:37.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.811294ms iteration = 152 status = {"config_updated":false}
232272025-11-13T08:11:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 77 reason = Timeout
232282025-11-13T08:11:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 153 reason = Timeout
232292025-11-13T08:11:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232302025-11-13T08:11:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
232312025-11-13T08:11:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
232322025-11-13T08:11:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
232332025-11-13T08:11:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232342025-11-13T08:11:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232352025-11-13T08:11:42.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
232362025-11-13T08:11:42.631ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232372025-11-13T08:11:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
232382025-11-13T08:11:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
232392025-11-13T08:11:42.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
232402025-11-13T08:11:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
232412025-11-13T08:11:42.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232422025-11-13T08:11:42.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
232432025-11-13T08:11:42.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
232442025-11-13T08:11:42.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.62502ms iteration = 153 status = {"config_updated":false}
232452025-11-13T08:11:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232462025-11-13T08:11:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
232472025-11-13T08:11:42.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
232482025-11-13T08:11:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
232492025-11-13T08:11:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 77 reason = Timeout
232502025-11-13T08:11:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
232512025-11-13T08:11:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
232522025-11-13T08:11:42.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
232532025-11-13T08:11:42.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
232542025-11-13T08:11:42.654ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
232552025-11-13T08:11:42.654ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.424799ms iteration = 77 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
232562025-11-13T08:11:42.677ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 34.317515ms iteration = 77 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
232572025-11-13T08:11:45.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 323 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = f77a9104-eb7f-4e70-b922-20992a33772c response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
232582025-11-13T08:11:47.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 67 reason = Timeout
232592025-11-13T08:11:47.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 154 reason = Timeout
232602025-11-13T08:11:47.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
232612025-11-13T08:11:47.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
232622025-11-13T08:11:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
232632025-11-13T08:11:47.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
232642025-11-13T08:11:47.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232652025-11-13T08:11:47.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232662025-11-13T08:11:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
232672025-11-13T08:11:47.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
232682025-11-13T08:11:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
232692025-11-13T08:11:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
232702025-11-13T08:11:47.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
232712025-11-13T08:11:47.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
232722025-11-13T08:11:47.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
232732025-11-13T08:11:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.950227ms iteration = 154 status = {"config_updated":false}
232742025-11-13T08:11:47.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3 original_time_started = 2025-11-13 08:11:03.994068 UTC
232752025-11-13T08:11:47.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.1428ms iteration = 67 status = {"Loaded":{"collection_id":"46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3","time_loaded":"2025-11-13T08:11:47.626099265Z","time_started":"2025-11-13T08:11:03.994068Z"}}
232762025-11-13T08:11:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 52 reason = Timeout
232772025-11-13T08:11:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
232782025-11-13T08:11:47.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 52 reason = Timeout
232792025-11-13T08:11:47.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
232802025-11-13T08:11:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
232812025-11-13T08:11:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
232822025-11-13T08:11:47.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
232832025-11-13T08:11:47.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
232842025-11-13T08:11:47.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232852025-11-13T08:11:47.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232862025-11-13T08:11:47.653ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
232872025-11-13T08:11:47.653ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
232882025-11-13T08:11:47.653ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
232892025-11-13T08:11:47.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
232902025-11-13T08:11:47.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.841485ms iteration = 52 status = "NoSitrep"
232912025-11-13T08:11:47.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
232922025-11-13T08:11:47.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
232932025-11-13T08:11:47.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
232942025-11-13T08:11:47.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
232952025-11-13T08:11:47.657ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
232962025-11-13T08:11:47.659ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
232972025-11-13T08:11:47.659ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
232982025-11-13T08:11:47.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.906095ms iteration = 52 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
232992025-11-13T08:11:52.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 78 reason = Timeout
233002025-11-13T08:11:52.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 155 reason = Timeout
233012025-11-13T08:11:52.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
233022025-11-13T08:11:52.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
233032025-11-13T08:11:52.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
233042025-11-13T08:11:52.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
233052025-11-13T08:11:52.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233062025-11-13T08:11:52.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233072025-11-13T08:11:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
233082025-11-13T08:11:52.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
233092025-11-13T08:11:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
233102025-11-13T08:11:52.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
233112025-11-13T08:11:52.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
233122025-11-13T08:11:52.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
233132025-11-13T08:11:52.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
233142025-11-13T08:11:52.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
233152025-11-13T08:11:52.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.272913ms iteration = 155 status = {"config_updated":false}
233162025-11-13T08:11:52.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
233172025-11-13T08:11:52.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233182025-11-13T08:11:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
233192025-11-13T08:11:52.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
233202025-11-13T08:11:52.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
233212025-11-13T08:11:52.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 78 reason = Timeout
233222025-11-13T08:11:52.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
233232025-11-13T08:11:52.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
233242025-11-13T08:11:52.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
233252025-11-13T08:11:52.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
233262025-11-13T08:11:52.653ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
233272025-11-13T08:11:52.653ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 26.616693ms iteration = 78 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
233282025-11-13T08:11:52.678ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 36.262023ms iteration = 78 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
233292025-11-13T08:11:55.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 432 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 54b23e3f-e8ae-49ff-b9cb-243ffb6c7ea3 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
233302025-11-13T08:11:57.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 156 reason = Timeout
233312025-11-13T08:11:57.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
233322025-11-13T08:11:57.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
233332025-11-13T08:11:57.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233342025-11-13T08:11:57.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
233352025-11-13T08:11:57.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
233362025-11-13T08:11:57.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
233372025-11-13T08:11:57.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
233382025-11-13T08:11:57.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.888455ms iteration = 156 status = {"config_updated":false}
233392025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 14 reason = Timeout
233402025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = external_endpoints iteration = 14 reason = Timeout
233412025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 27 reason = Timeout
233422025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
233432025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 16 reason = Timeout
233442025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 16 reason = Timeout
233452025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 27 reason = Timeout
233462025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 14 reason = Timeout
233472025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
233482025-11-13T08:12:02.607ZINFOnexus (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-11-13 08:02:02.607371525 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:48
233492025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 27 reason = Timeout
233502025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
233512025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
233522025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 14 reason = Timeout
233532025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 14 reason = Timeout
233542025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 14 reason = Timeout
233552025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
233562025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
233572025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
233582025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
233592025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_executor iteration = 15 reason = Timeout
233602025-11-13T08:12:02.607ZDEBGnexus (BackgroundTasks): blueprint execution: quiesce check background_task = blueprint_executor quiescing = false
233612025-11-13T08:12:02.607ZINFOnexus (BackgroundTasks): attempting to realize blueprint background_task = blueprint_executor blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/lib.rs:180
233622025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
233632025-11-13T08:12:02.608ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = ExternalNetworking step id = Ensure
233642025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
233652025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
233662025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
233672025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
233682025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
233692025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
233702025-11-13T08:12:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("0b826b69-7e7f-4662-88f8-01d591c1a724.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("353e9780-feb1-419a-a764-07869c5904de.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("55b55228-bf82-4bcb-b680-ded78e45bf10.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:1::1)) }], valid_until: Instant { tv_sec: 6730, tv_nsec: 960311035 } })
233712025-11-13T08:12:02.608ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("25561cfb-881a-4ed6-9d55-6d9461030a3d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 5353, target: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("bc3a50d3-9262-410c-8c88-068c994facb4.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::9)) }], valid_until: Instant { tv_sec: 6730, tv_nsec: 960327965 } })
233722025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
233732025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } background_task = dns_servers_internal current_servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } dns_group = internal
233742025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 1.50007ms iteration = 14 status = {"addresses":["[fd00:1122:3344:2::1]:5353","[fd00:1122:3344:3::1]:5353","[fd00:1122:3344:1::1]:5353"]}
233752025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): found DNS servers (no change) addresses = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } background_task = dns_servers_external current_servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } dns_group = external
233762025-11-13T08:12:02.608ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 1.733274ms iteration = 14 status = {"addresses":["[fd00:1122:3344:101::8]:5353","[fd00:1122:3344:101::9]:5353"]}
233772025-11-13T08:12:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
233782025-11-13T08:12:02.609ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
233792025-11-13T08:12:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
233802025-11-13T08:12:02.610ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
233812025-11-13T08:12:02.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6730, tv_nsec: 963436806 } })
233822025-11-13T08:12:02.612ZINFOnexus (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-11-13 08:02:02.607371525 UTC file = nexus/src/app/background/tasks/metrics_producer_gc.rs:69 pruned = {}
233832025-11-13T08:12:02.612ZDEBGnexus (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 4.711592ms iteration = 14 status = {"expiration":"2025-11-13T08:02:02.607371525Z","pruned":[]}
233842025-11-13T08:12:02.613ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233852025-11-13T08:12:02.613ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6730, tv_nsec: 965536688 } })
233862025-11-13T08:12:02.613ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
233872025-11-13T08:12:02.614ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233882025-11-13T08:12:02.616ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
233892025-11-13T08:12:02.617ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
233902025-11-13T08:12:02.617ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
233912025-11-13T08:12:02.617ZDEBGnexus (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 14 reason = Timeout
233922025-11-13T08:12:02.617ZINFOnexus (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:52
233932025-11-13T08:12:02.617ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
233942025-11-13T08:12:02.618ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 32 reason = Timeout
233952025-11-13T08:12:02.618ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
233962025-11-13T08:12:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
233972025-11-13T08:12:02.618ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
233982025-11-13T08:12:02.619ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
233992025-11-13T08:12:02.619ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
234002025-11-13T08:12:02.619ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
234012025-11-13T08:12:02.620ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
234022025-11-13T08:12:02.620ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234032025-11-13T08:12:02.621ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234042025-11-13T08:12:02.621ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
234052025-11-13T08:12:02.621ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
234062025-11-13T08:12:02.621ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
234072025-11-13T08:12:02.621ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
234082025-11-13T08:12:02.621ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
234092025-11-13T08:12:02.624ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
234102025-11-13T08:12:02.624ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
234112025-11-13T08:12:02.624ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234122025-11-13T08:12:02.624ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
234132025-11-13T08:12:02.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
234142025-11-13T08:12:02.624ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234152025-11-13T08:12:02.625ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
234162025-11-13T08:12:02.625ZDEBGnexus (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
234172025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
234182025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 40 reason = Timeout
234192025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
234202025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 68 reason = Timeout
234212025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 43 reason = Timeout
234222025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 157 reason = Timeout
234232025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 79 reason = Timeout
234242025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
234252025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
234262025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
234272025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
234282025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
234292025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
234302025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
234312025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
234322025-11-13T08:12:02.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
234332025-11-13T08:12:02.627ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234342025-11-13T08:12:02.629ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234352025-11-13T08:12:02.629ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234362025-11-13T08:12:02.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234372025-11-13T08:12:02.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234382025-11-13T08:12:02.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234392025-11-13T08:12:02.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
234402025-11-13T08:12:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
234412025-11-13T08:12:02.634ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
234422025-11-13T08:12:02.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
234432025-11-13T08:12:02.634ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
234442025-11-13T08:12:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
234452025-11-13T08:12:02.635ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 27.907558ms iteration = 27 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
234462025-11-13T08:12:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
234472025-11-13T08:12:02.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
234482025-11-13T08:12:02.635ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
234492025-11-13T08:12:02.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
234502025-11-13T08:12:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
234512025-11-13T08:12:02.637ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
234522025-11-13T08:12:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
234532025-11-13T08:12:02.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
234542025-11-13T08:12:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
234552025-11-13T08:12:02.637ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
234562025-11-13T08:12:02.638ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
234572025-11-13T08:12:02.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
234582025-11-13T08:12:02.639ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
234592025-11-13T08:12:02.640ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
234602025-11-13T08:12:02.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
234612025-11-13T08:12:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
234622025-11-13T08:12:02.640ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
234632025-11-13T08:12:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
234642025-11-13T08:12:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
234652025-11-13T08:12:02.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
234662025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 14 reason = Timeout
234672025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
234682025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 40 reason = Timeout
234692025-11-13T08:12:02.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
234702025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 27 reason = Timeout
234712025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
234722025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
234732025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 27 reason = Timeout
234742025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
234752025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
234762025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
234772025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
234782025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 27 reason = Timeout
234792025-11-13T08:12:02.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
234802025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
234812025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
234822025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 79 reason = Timeout
234832025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 27 reason = Timeout
234842025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
234852025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
234862025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 27 reason = Timeout
234872025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
234882025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
234892025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_reincarnation iteration = 14 reason = Timeout
234902025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
234912025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 27 reason = Timeout
234922025-11-13T08:12:02.643ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
234932025-11-13T08:12:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
234942025-11-13T08:12:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
234952025-11-13T08:12:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
234962025-11-13T08:12:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
234972025-11-13T08:12:02.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
234982025-11-13T08:12:02.645ZDEBGnexus (BackgroundTasks): activating background_task = lookup_region_port iteration = 14 reason = Timeout
234992025-11-13T08:12:02.645ZDEBGnexus (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
235002025-11-13T08:12:02.646ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = lookup_region_port resource = Database result = Ok(())
235012025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = webhook_deliverator iteration = 14 reason = Timeout
235022025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 27 reason = Timeout
235032025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
235042025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
235052025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 27 reason = Timeout
235062025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 53 reason = Timeout
235072025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
235082025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 53 reason = Timeout
235092025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
235102025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 27 reason = Timeout
235112025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
235122025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 27 reason = Timeout
235132025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
235142025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 27 reason = Timeout
235152025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 27 reason = Timeout
235162025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
235172025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
235182025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
235192025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
235202025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = webhook_deliverator resource = Database result = Ok(())
235212025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
235222025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activating background_task = alert_dispatcher iteration = 14 reason = Timeout
235232025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
235242025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
235252025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 22.675076ms iteration = 157 status = {"config_updated":false}
235262025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
235272025-11-13T08:12:02.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
235282025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
235292025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): activating background_task = probe_distributor iteration = 14 reason = Timeout
235302025-11-13T08:12:02.649ZINFOnexus (BackgroundTasks): distributing networking probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:44
235312025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
235322025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
235332025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
235342025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
235352025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
235362025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235372025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
235382025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
235392025-11-13T08:12:02.649ZINFOnexus (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper file = nexus/src/app/background/tasks/abandoned_vmm_reaper.rs:170 sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
235402025-11-13T08:12:02.649ZDEBGnexus (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 7.236722ms iteration = 14 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
235412025-11-13T08:12:02.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6731, tv_nsec: 2078736 } })
235422025-11-13T08:12:02.652ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235432025-11-13T08:12:02.653ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235442025-11-13T08:12:02.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
235452025-11-13T08:12:02.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
235462025-11-13T08:12:02.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
235472025-11-13T08:12:02.656ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
235482025-11-13T08:12:02.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
235492025-11-13T08:12:02.657ZDEBGnexus (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 12.086698ms iteration = 14 status = {"errors":[],"found_port_ok":[]}
235502025-11-13T08:12:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
235512025-11-13T08:12:02.658ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
235522025-11-13T08:12:02.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
235532025-11-13T08:12:02.659ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
235542025-11-13T08:12:02.659ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
235552025-11-13T08:12:02.659ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235562025-11-13T08:12:02.659ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
235572025-11-13T08:12:02.663ZINFOnexus (BackgroundTasks): all webhook delivery tasks completed background_task = webhook_deliverator file = nexus/src/app/background/tasks/webhook_deliverator.rs:204 num_receivers = 0
235582025-11-13T08:12:02.663ZDEBGnexus (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 14.850471ms iteration = 14 status = {"by_rx":{},"error":null}
235592025-11-13T08:12:02.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
235602025-11-13T08:12:02.663ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
235612025-11-13T08:12:02.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
235622025-11-13T08:12:02.664ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3 original_time_started = 2025-11-13 08:11:03.994068 UTC
235632025-11-13T08:12:02.664ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 38.037768ms iteration = 68 status = {"Loaded":{"collection_id":"46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3","time_loaded":"2025-11-13T08:12:02.626187195Z","time_started":"2025-11-13T08:11:03.994068Z"}}
235642025-11-13T08:12:02.664ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 16.24934ms iteration = 27 status = {"errors":[],"garbage_collect_requested":[]}
235652025-11-13T08:12:02.668ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
235662025-11-13T08:12:02.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
235672025-11-13T08:12:02.669ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
235682025-11-13T08:12:02.669ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
235692025-11-13T08:12:02.679ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 30.693204ms iteration = 27 status = {"errors":[],"requests_created_ok":[]}
235702025-11-13T08:12:02.684ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235712025-11-13T08:12:02.687ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235722025-11-13T08:12:02.688ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
235732025-11-13T08:12:02.689ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
235742025-11-13T08:12:02.689ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
235752025-11-13T08:12:02.693ZDEBGnexus (BackgroundTasks): roles background_task = alert_dispatcher roles = RoleSet { roles: {} }
235762025-11-13T08:12:02.693ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
235772025-11-13T08:12:02.693ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
235782025-11-13T08:12:02.693ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
235792025-11-13T08:12:02.694ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = alert_dispatcher resource = Database result = Ok(())
235802025-11-13T08:12:02.696ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6731, tv_nsec: 49071160 } })
235812025-11-13T08:12:02.697ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
235822025-11-13T08:12:02.712ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235832025-11-13T08:12:02.716ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
235842025-11-13T08:12:02.719ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
235852025-11-13T08:12:02.719ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 76.153477ms iteration = 27 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
235862025-11-13T08:12:02.721ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
235872025-11-13T08:12:02.723ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 75.115727ms iteration = 53 status = "NoSitrep"
235882025-11-13T08:12:02.724ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
235892025-11-13T08:12:02.725ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
235902025-11-13T08:12:02.726ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235912025-11-13T08:12:02.726ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: e08dba8b-d44d-4d6b-9a29-22e2146c33de (external_ip), ip: 10.151.2.164 }) nic = NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 25561cfb-881a-4ed6-9d55-6d9461030a3d zone_kind = external_dns
235922025-11-13T08:12:02.728ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
235932025-11-13T08:12:02.728ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
235942025-11-13T08:12:02.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
235952025-11-13T08:12:02.732ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
235962025-11-13T08:12:02.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
235972025-11-13T08:12:02.734ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
235982025-11-13T08:12:02.734ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 92.03785ms iteration = 27 status = 0
235992025-11-13T08:12:02.735ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
236002025-11-13T08:12:02.735ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
236012025-11-13T08:12:02.736ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236022025-11-13T08:12:02.738ZDEBGnexus (BackgroundTasks): activation complete background_task = alert_dispatcher elapsed = 89.342507ms iteration = 14 status = {"dispatched":[],"errors":[],"glob_version":"207.0.0","globs_reprocessed":{},"no_receivers":[]}
236032025-11-13T08:12:02.738ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 59be305b-caf2-437a-9506-dd4dbd32ba37 (external_ip), ip: 10.151.2.167 }) nic = NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 6564a4a0-ff58-4081-9860-74f24e488435 zone_kind = nexus
236042025-11-13T08:12:02.739ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
236052025-11-13T08:12:02.739ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
236062025-11-13T08:12:02.740ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
236072025-11-13T08:12:02.740ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
236082025-11-13T08:12:02.740ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
236092025-11-13T08:12:02.741ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
236102025-11-13T08:12:02.743ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 89bd89fb-ec69-4a70-aeb2-8599f6e9a894 (external_ip), ip: 10.151.2.168 }) nic = NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 79428d63-073a-4ba3-9687-ed29ef6cec3a zone_kind = nexus
236112025-11-13T08:12:02.745ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
236122025-11-13T08:12:02.746ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
236132025-11-13T08:12:02.746ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
236142025-11-13T08:12:02.746ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
236152025-11-13T08:12:02.746ZDEBGnexus (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
236162025-11-13T08:12:02.746ZDEBGnexus (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
236172025-11-13T08:12:02.746ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
236182025-11-13T08:12:02.746ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
236192025-11-13T08:12:02.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
236202025-11-13T08:12:02.747ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
236212025-11-13T08:12:02.747ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Snat(OmicronZoneExternalSnatIp { id: 2d859ebc-843b-40e5-a64c-98b427cb0180 (external_ip), snat_cfg: SourceNatConfig { ip: 10.151.2.169, first_port: 0, last_port: 16383 } }) nic = NetworkInterface { id: f3cec8e2-2aa0-4c05-84a3-7c1919dfe74b, kind: Service { id: 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 }, name: Name("ntp-83a0d63c-a7a2-4e4a-aeca-da01f3d52209"), ip: 172.30.3.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 183, 194])), subnet: V4(Ipv4Net { addr: 172.30.3.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 zone_kind = boundary_ntp
236222025-11-13T08:12:02.748ZDEBGnexus (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
236232025-11-13T08:12:02.748ZINFOnexus (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 file = nexus/src/app/background/tasks/decommissioned_disk_cleaner.rs:137 found = 0 not_ready_to_be_deleted = 0
236242025-11-13T08:12:02.748ZDEBGnexus (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 130.589517ms iteration = 14 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
236252025-11-13T08:12:02.748ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236262025-11-13T08:12:02.750ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
236272025-11-13T08:12:02.750ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236282025-11-13T08:12:02.751ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236292025-11-13T08:12:02.751ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236302025-11-13T08:12:02.752ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
236312025-11-13T08:12:02.752ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: 266d569b-ad67-4ce2-b876-5446c390779b (external_ip), ip: 10.151.2.165 }) nic = NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bc3a50d3-9262-410c-8c88-068c994facb4 zone_kind = external_dns
236322025-11-13T08:12:02.753ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
236332025-11-13T08:12:02.753ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 104.656817ms iteration = 53 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
236342025-11-13T08:12:02.754ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
236352025-11-13T08:12:02.754ZINFOnexus (BackgroundTasks): found already-allocated external IP action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:261 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
236362025-11-13T08:12:02.754ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
236372025-11-13T08:12:02.756ZINFOnexus (BackgroundTasks): found already-allocated NIC action = allocate-external-networking background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/deployment/external_networking.rs:322 ip = Floating(OmicronZoneExternalFloatingIp { id: d4be9da8-6297-48a9-84ef-61cde73e959c (external_ip), ip: 10.151.2.166 }) nic = NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] } zone_id = bcb58129-79cc-406c-aa16-455a5cd9f031 zone_kind = nexus
236382025-11-13T08:12:02.757ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = SledList step id = Fetch
236392025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236402025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
236412025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
236422025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
236432025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
236442025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
236452025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236462025-11-13T08:12:02.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
236472025-11-13T08:12:02.758ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
236482025-11-13T08:12:02.758ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
236492025-11-13T08:12:02.758ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
236502025-11-13T08:12:02.759ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
236512025-11-13T08:12:02.762ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
236522025-11-13T08:12:02.762ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
236532025-11-13T08:12:02.762ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
236542025-11-13T08:12:02.765ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236552025-11-13T08:12:02.765ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236562025-11-13T08:12:02.766ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
236572025-11-13T08:12:02.766ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
236582025-11-13T08:12:02.769ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
236592025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
236602025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
236612025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
236622025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
236632025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
236642025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
236652025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_external current_generation = 2 current_time_created = 2025-11-13 07:59:13.416569 UTC dns_group = external generation = 2
236662025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 162.74761ms iteration = 14 status = {"generation":2}
236672025-11-13T08:12:02.770ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
236682025-11-13T08:12:02.787ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 139.570024ms iteration = 27 status = {"errors":[],"finish_invoked_ok":[]}
236692025-11-13T08:12:02.795ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
236702025-11-13T08:12:02.795ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
236712025-11-13T08:12:02.795ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
236722025-11-13T08:12:02.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
236732025-11-13T08:12:02.800ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
236742025-11-13T08:12:02.800ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
236752025-11-13T08:12:02.800ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
236762025-11-13T08:12:02.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
236772025-11-13T08:12:02.801ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
236782025-11-13T08:12:02.801ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
236792025-11-13T08:12:02.805ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 162.644398ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
236802025-11-13T08:12:02.809ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
236812025-11-13T08:12:02.809ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
236822025-11-13T08:12:02.815ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
236832025-11-13T08:12:02.815ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 189.297701ms iteration = 79 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
236842025-11-13T08:12:02.834ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::8]:5353/config
236852025-11-13T08:12:02.839ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
236862025-11-13T08:12:02.839ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 213.244902ms iteration = 40 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
236872025-11-13T08:12:02.839ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
236882025-11-13T08:12:02.839ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
236892025-11-13T08:12:02.840ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "939db81d-70e0-4573-b20a-e0d5c524014a", "content-length": "78", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} })
236902025-11-13T08:12:02.840ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
236912025-11-13T08:12:02.840ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
236922025-11-13T08:12:02.840ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
236932025-11-13T08:12:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
236942025-11-13T08:12:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
236952025-11-13T08:12:02.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
236962025-11-13T08:12:02.842ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
236972025-11-13T08:12:02.843ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
236982025-11-13T08:12:02.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
236992025-11-13T08:12:02.844ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 200.810939ms iteration = 79 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
237002025-11-13T08:12:02.844ZDEBGnexus (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
237012025-11-13T08:12:02.844ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
237022025-11-13T08:12:02.844ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
237032025-11-13T08:12:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237042025-11-13T08:12:02.844ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
237052025-11-13T08:12:02.846ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
237062025-11-13T08:12:02.846ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
237072025-11-13T08:12:02.847ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
237082025-11-13T08:12:02.847ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
237092025-11-13T08:12:02.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237102025-11-13T08:12:02.850ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
237112025-11-13T08:12:02.851ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
237122025-11-13T08:12:02.851ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
237132025-11-13T08:12:02.851ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
237142025-11-13T08:12:02.852ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237152025-11-13T08:12:02.852ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 204.403208ms iteration = 27 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
237162025-11-13T08:12:02.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
237172025-11-13T08:12:02.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237182025-11-13T08:12:02.853ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237192025-11-13T08:12:02.853ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 246.109529ms iteration = 27 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
237202025-11-13T08:12:02.857ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
237212025-11-13T08:12:02.857ZDEBGnexus (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
237222025-11-13T08:12:02.857ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
237232025-11-13T08:12:02.857ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = DeployNexusRecords step id = Ensure
237242025-11-13T08:12:02.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237252025-11-13T08:12:02.858ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237262025-11-13T08:12:02.861ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
237272025-11-13T08:12:02.861ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
237282025-11-13T08:12:02.861ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
237292025-11-13T08:12:02.861ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
237302025-11-13T08:12:02.861ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 218.773631ms iteration = 27 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
237312025-11-13T08:12:02.862ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e547a63-22bd-4892-91a3-3d316202b4fd", "content-length": "26", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} })
237322025-11-13T08:12:02.862ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
237332025-11-13T08:12:02.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237342025-11-13T08:12:02.863ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237352025-11-13T08:12:02.864ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
237362025-11-13T08:12:02.864ZDEBGnexus (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
237372025-11-13T08:12:02.864ZDEBGnexus (BackgroundTasks): found latest DNS generation (unchanged) background_task = dns_config_internal current_generation = 1 current_time_created = 2025-11-13 07:59:12.439669 UTC dns_group = internal generation = 1
237382025-11-13T08:12:02.864ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 257.463072ms iteration = 14 status = {"generation":1}
237392025-11-13T08:12:02.865ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
237402025-11-13T08:12:02.866ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
237412025-11-13T08:12:02.866ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237422025-11-13T08:12:02.866ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237432025-11-13T08:12:02.870ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
237442025-11-13T08:12:02.870ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Fleet result = Ok(())
237452025-11-13T08:12:02.870ZDEBGnexus (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
237462025-11-13T08:12:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
237472025-11-13T08:12:02.870ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = external_endpoints resource = Database result = Ok(())
237482025-11-13T08:12:02.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
237492025-11-13T08:12:02.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
237502025-11-13T08:12:02.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237512025-11-13T08:12:02.877ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237522025-11-13T08:12:02.881ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
237532025-11-13T08:12:02.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
237542025-11-13T08:12:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
237552025-11-13T08:12:02.882ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
237562025-11-13T08:12:02.882ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
237572025-11-13T08:12:02.883ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
237582025-11-13T08:12:02.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237592025-11-13T08:12:02.886ZDEBGnexus (BackgroundTasks): found Silo/DNS/TLS config (no change) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-11-13T07:59:00.213874Z, time_modified: 2025-11-13T07:59:00.213874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)), admin_group_name: None }, tls_certs: [] }, "recovery.sys.oxide.test": ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 98277b66-8496-4a1b-9f11-9c6ba8511252, db_silo: Silo { identity: SiloIdentity { id: 98277b66-8496-4a1b-9f11-9c6ba8511252, name: Name(Name("recovery")), description: "built-in recovery Silo", time_created: 2025-11-13T07:59:13.355270Z, time_modified: 2025-11-13T07:59:13.355270Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(4)), admin_group_name: None }, tls_certs: [TlsCertificate { digest: "d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38" }] }) }
237602025-11-13T08:12:02.886ZDEBGnexus (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 279.287431ms iteration = 14 status = {"by_dns_name":{"default-silo.sys.oxide.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"recovery.sys.oxide.test":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]}},"default_endpoint":{"silo_id":"98277b66-8496-4a1b-9f11-9c6ba8511252","tls_certs":["d9490525a54dcb3691f2f08172a61950255e76a344d313be96fd8c3b5f4c0e38"]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide.test\\" has no usable certificates"]}
237612025-11-13T08:12:02.887ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
237622025-11-13T08:12:02.887ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
237632025-11-13T08:12:02.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
237642025-11-13T08:12:02.893ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:2::1]:5353/config
237652025-11-13T08:12:02.897ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
237662025-11-13T08:12:02.897ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
237672025-11-13T08:12:02.897ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
237682025-11-13T08:12:02.898ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "2437634d-3f37-44ad-8ff0-303db1080dfb", "content-length": "26", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} })
237692025-11-13T08:12:02.898ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
237702025-11-13T08:12:02.898ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
237712025-11-13T08:12:02.899ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237722025-11-13T08:12:02.900ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
237732025-11-13T08:12:02.903ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Fleet result = Ok(())
237742025-11-13T08:12:02.903ZDEBGnexus (BackgroundTasks): roles background_task = probe_distributor roles = RoleSet { roles: {} }
237752025-11-13T08:12:02.903ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = probe_distributor resource = Database result = Ok(())
237762025-11-13T08:12:02.908ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
237772025-11-13T08:12:02.911ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
237782025-11-13T08:12:02.911ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
237792025-11-13T08:12:02.911ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
237802025-11-13T08:12:02.922ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
237812025-11-13T08:12:02.922ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
237822025-11-13T08:12:02.923ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "71b8cf56-5eff-4474-8e39-399520cfdb3a", "content-length": "81", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} })
237832025-11-13T08:12:02.923ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
237842025-11-13T08:12:02.924ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
237852025-11-13T08:12:02.934ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
237862025-11-13T08:12:02.934ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
237872025-11-13T08:12:02.934ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "993190ef-da12-4e25-a00b-898e0ac0c6d6", "content-length": "81", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} })
237882025-11-13T08:12:02.940ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
237892025-11-13T08:12:02.941ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
237902025-11-13T08:12:02.943ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237912025-11-13T08:12:02.944ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = SledAgent step id = Ensure
237922025-11-13T08:12:02.945ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 297.58289ms iteration = 27 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
237932025-11-13T08:12:02.946ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
237942025-11-13T08:12:02.947ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
237952025-11-13T08:12:02.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
237962025-11-13T08:12:02.948ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
237972025-11-13T08:12:02.949ZDEBGnexus (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
237982025-11-13T08:12:02.949ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_reincarnation resource = Database result = Ok(())
237992025-11-13T08:12:02.952ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
238002025-11-13T08:12:02.952ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
238012025-11-13T08:12:02.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
238022025-11-13T08:12:02.954ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238032025-11-13T08:12:02.955ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238042025-11-13T08:12:02.958ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238052025-11-13T08:12:02.958ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238062025-11-13T08:12:02.959ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238072025-11-13T08:12:02.966ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
238082025-11-13T08:12:02.967ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238092025-11-13T08:12:02.967ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238102025-11-13T08:12:02.969ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238112025-11-13T08:12:02.969ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238122025-11-13T08:12:02.971ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] } uri = http://[fd00:1122:3344:101::9]:5353/config
238132025-11-13T08:12:02.971ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::9]:5353/config", status: 204, headers: {"x-request-id": "b6a75d2f-1480-41ec-9622-64f9aa6edc70", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
238142025-11-13T08:12:02.971ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::9]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
238152025-11-13T08:12:02.972ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238162025-11-13T08:12:02.974ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 generation = 2 result = Ok(Response { url: "http://[fd00:1122:3344:101::8]:5353/config", status: 204, headers: {"x-request-id": "31a0fe69-af0f-4b47-9edd-321d275bbcc9", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
238172025-11-13T08:12:02.974ZINFOnexus (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[fd00:1122:3344:101::8]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
238182025-11-13T08:12:02.979ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
238192025-11-13T08:12:02.979ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238202025-11-13T08:12:02.980ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238212025-11-13T08:12:02.980ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
238222025-11-13T08:12:02.980ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
238232025-11-13T08:12:02.981ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d84561c-bde2-4155-b05b-7f132bd813ce", "content-length": "81", "date": "Thu, 13 Nov 2025 08:12:02 GMT"} })
238242025-11-13T08:12:02.982ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 2 servers = DnsServersList { addresses: [[fd00:1122:3344:101::8]:5353, [fd00:1122:3344:101::9]:5353] }
238252025-11-13T08:12:02.982ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 374.946742ms iteration = 16 status = {"generation":2,"server_results":{"[fd00:1122:3344:101::8]:5353":{"Ok":null},"[fd00:1122:3344:101::9]:5353":{"Ok":null}}}
238262025-11-13T08:12:02.983ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238272025-11-13T08:12:02.985ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 336.941934ms iteration = 27 status = {"disabled":false,"errors":[],"sps":[]}
238282025-11-13T08:12:02.987ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 361.611209ms iteration = 43 status = {"physical_disks_added":0}
238292025-11-13T08:12:02.988ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
238302025-11-13T08:12:02.988ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238312025-11-13T08:12:02.988ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238322025-11-13T08:12:02.992ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238332025-11-13T08:12:02.993ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238342025-11-13T08:12:02.995ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238352025-11-13T08:12:02.999ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
238362025-11-13T08:12:02.999ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238372025-11-13T08:12:02.999ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238382025-11-13T08:12:03.002ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238392025-11-13T08:12:03.002ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238402025-11-13T08:12:03.004ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238412025-11-13T08:12:03.007ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
238422025-11-13T08:12:03.007ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238432025-11-13T08:12:03.007ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238442025-11-13T08:12:03.009ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238452025-11-13T08:12:03.009ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238462025-11-13T08:12:03.010ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238472025-11-13T08:12:03.015ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
238482025-11-13T08:12:03.015ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238492025-11-13T08:12:03.016ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238502025-11-13T08:12:03.032ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6731, tv_nsec: 384739379 } })
238512025-11-13T08:12:03.045ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238522025-11-13T08:12:03.046ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238532025-11-13T08:12:03.049ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238542025-11-13T08:12:03.050ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238552025-11-13T08:12:03.051ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238562025-11-13T08:12:03.059ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
238572025-11-13T08:12:03.059ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238582025-11-13T08:12:03.059ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238592025-11-13T08:12:03.062ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238602025-11-13T08:12:03.065ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
238612025-11-13T08:12:03.065ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238622025-11-13T08:12:03.066ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238632025-11-13T08:12:03.068ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238642025-11-13T08:12:03.068ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238652025-11-13T08:12:03.070ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238662025-11-13T08:12:03.071ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238672025-11-13T08:12:03.071ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 463.794898ms iteration = 27 status = {}
238682025-11-13T08:12:03.073ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238692025-11-13T08:12:03.076ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
238702025-11-13T08:12:03.076ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238712025-11-13T08:12:03.077ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238722025-11-13T08:12:03.079ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238732025-11-13T08:12:03.083ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
238742025-11-13T08:12:03.083ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238752025-11-13T08:12:03.083ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238762025-11-13T08:12:03.086ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238772025-11-13T08:12:03.087ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238782025-11-13T08:12:03.087ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238792025-11-13T08:12:03.088ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238802025-11-13T08:12:03.089ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238812025-11-13T08:12:03.094ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
238822025-11-13T08:12:03.107ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/probes
238832025-11-13T08:12:03.108ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = probe_distributor result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/probes", status: 204, headers: {"x-request-id": "d920b046-75da-48d6-ab37-75c0c8518d3a", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
238842025-11-13T08:12:03.108ZDEBGnexus (BackgroundTasks): Update set of probes on sled background_task = probe_distributor n_probes = 0 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
238852025-11-13T08:12:03.109ZINFOnexus (BackgroundTasks): finished distributing probes to sleds background_task = probe_distributor file = nexus/src/app/background/tasks/probe_distributor.rs:133 n_pushed_probes = 0
238862025-11-13T08:12:03.109ZDEBGnexus (BackgroundTasks): activation complete background_task = probe_distributor elapsed = 460.880011ms iteration = 14 status = {"errors":[],"probes_by_sled":{"370ec2da-8f7f-4861-9b77-369c2bb1c86d":0}}
238872025-11-13T08:12:03.129ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
238882025-11-13T08:12:03.129ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
238892025-11-13T08:12:03.130ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "34fc1fde-5f15-4b00-99cd-21240a4068ad", "content-length": "372", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
238902025-11-13T08:12:03.130ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
238912025-11-13T08:12:03.130ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
238922025-11-13T08:12:03.132ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238932025-11-13T08:12:03.132ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238942025-11-13T08:12:03.133ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
238952025-11-13T08:12:03.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
238962025-11-13T08:12:03.134ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
238972025-11-13T08:12:03.139ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
238982025-11-13T08:12:03.167ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:3::1]:5353/config
238992025-11-13T08:12:03.168ZINFOnexus (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation file = nexus/src/app/background/tasks/instance_reincarnation.rs:113 instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
239002025-11-13T08:12:03.168ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 525.177745ms iteration = 14 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
239012025-11-13T08:12:03.181ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
239022025-11-13T08:12:03.182ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
239032025-11-13T08:12:03.184ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
239042025-11-13T08:12:03.185ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
239052025-11-13T08:12:03.188ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
239062025-11-13T08:12:03.188ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
239072025-11-13T08:12:03.188ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "fb407f45-734f-4387-ba27-385bb33f897d", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
239082025-11-13T08:12:03.188ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239092025-11-13T08:12:03.188ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239102025-11-13T08:12:03.189ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
239112025-11-13T08:12:03.189ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bbee6e6-250d-4448-82dd-fe82b739ad8d", "content-length": "691", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
239122025-11-13T08:12:03.189ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
239132025-11-13T08:12:03.189ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
239142025-11-13T08:12:03.189ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
239152025-11-13T08:12:03.191ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239162025-11-13T08:12:03.191ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239172025-11-13T08:12:03.192ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 548.908222ms iteration = 27 status = {}
239182025-11-13T08:12:03.194ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239192025-11-13T08:12:03.194ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239202025-11-13T08:12:03.197ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup generation = 5 method = PUT sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d uri = http://[fd00:1122:3344:101::1]:12345/omicron-config
239212025-11-13T08:12:03.198ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
239222025-11-13T08:12:03.198ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
239232025-11-13T08:12:03.198ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "dc8bb173-3ccc-498b-b792-355095299f20", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
239242025-11-13T08:12:03.199ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup generation = 5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/omicron-config", status: 204, headers: {"x-request-id": "e673d268-22f0-4c90-be55-ac7e86b0bab2", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} }) sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
239252025-11-13T08:12:03.201ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 558.724584ms iteration = 40 status = {}
239262025-11-13T08:12:03.201ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 41 reason = Dependency
239272025-11-13T08:12:03.201ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
239282025-11-13T08:12:03.201ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239292025-11-13T08:12:03.201ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239302025-11-13T08:12:03.201ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = FirewallRules step id = Ensure
239312025-11-13T08:12:03.202ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239322025-11-13T08:12:03.202ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239332025-11-13T08:12:03.203ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239342025-11-13T08:12:03.204ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239352025-11-13T08:12:03.204ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239362025-11-13T08:12:03.205ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
239372025-11-13T08:12:03.205ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239382025-11-13T08:12:03.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239392025-11-13T08:12:03.206ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239402025-11-13T08:12:03.206ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239412025-11-13T08:12:03.207ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239422025-11-13T08:12:03.207ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239432025-11-13T08:12:03.208ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239442025-11-13T08:12:03.209ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
239452025-11-13T08:12:03.209ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
239462025-11-13T08:12:03.209ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
239472025-11-13T08:12:03.222ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239482025-11-13T08:12:03.225ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
239492025-11-13T08:12:03.225ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
239502025-11-13T08:12:03.226ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
239512025-11-13T08:12:03.228ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
239522025-11-13T08:12:03.228ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
239532025-11-13T08:12:03.230ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239542025-11-13T08:12:03.233ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
239552025-11-13T08:12:03.233ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
239562025-11-13T08:12:03.233ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
239572025-11-13T08:12:03.236ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
239582025-11-13T08:12:03.236ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
239592025-11-13T08:12:03.254ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239602025-11-13T08:12:03.254ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239612025-11-13T08:12:03.256ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239622025-11-13T08:12:03.256ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239632025-11-13T08:12:03.257ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239642025-11-13T08:12:03.260ZDEBGnexus (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 method = PUT servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] } uri = http://[fd00:1122:3344:1::1]:5353/config
239652025-11-13T08:12:03.260ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
239662025-11-13T08:12:03.260ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
239672025-11-13T08:12:03.260ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 204, headers: {"x-request-id": "1949c21e-f2cb-4f00-99da-cd6da14513aa", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
239682025-11-13T08:12:03.260ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:1::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
239692025-11-13T08:12:03.262ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
239702025-11-13T08:12:03.262ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239712025-11-13T08:12:03.262ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
239722025-11-13T08:12:03.262ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
239732025-11-13T08:12:03.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239742025-11-13T08:12:03.262ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
239752025-11-13T08:12:03.262ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 204, headers: {"x-request-id": "11298d6b-1133-4bc1-bc29-cb02aa5e0b4e", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
239762025-11-13T08:12:03.262ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:3::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
239772025-11-13T08:12:03.263ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239782025-11-13T08:12:03.263ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239792025-11-13T08:12:03.264ZDEBGnexus (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 generation = 1 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 204, headers: {"x-request-id": "1d12a5ad-1e35-4eb5-a503-f2a30695c5af", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} }) servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
239802025-11-13T08:12:03.264ZINFOnexus (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[fd00:1122:3344:2::1]:5353 file = nexus/src/app/background/tasks/dns_propagation.rs:169 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
239812025-11-13T08:12:03.265ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239822025-11-13T08:12:03.265ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239832025-11-13T08:12:03.266ZINFOnexus (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal file = nexus/src/app/background/tasks/dns_propagation.rs:106 generation = 1 servers = DnsServersList { addresses: [[fd00:1122:3344:2::1]:5353, [fd00:1122:3344:3::1]:5353, [fd00:1122:3344:1::1]:5353] }
239842025-11-13T08:12:03.266ZDEBGnexus (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 659.525526ms iteration = 16 status = {"generation":1,"server_results":{"[fd00:1122:3344:1::1]:5353":{"Ok":null},"[fd00:1122:3344:2::1]:5353":{"Ok":null},"[fd00:1122:3344:3::1]:5353":{"Ok":null}}}
239852025-11-13T08:12:03.266ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
239862025-11-13T08:12:03.273ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
239872025-11-13T08:12:03.273ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
239882025-11-13T08:12:03.274ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
239892025-11-13T08:12:03.278ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
239902025-11-13T08:12:03.278ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
239912025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
239922025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
239932025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239942025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239952025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239962025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
239972025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
239982025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
239992025-11-13T08:12:03.279ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
240002025-11-13T08:12:03.279ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
240012025-11-13T08:12:03.279ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240022025-11-13T08:12:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
240032025-11-13T08:12:03.280ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240042025-11-13T08:12:03.283ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240052025-11-13T08:12:03.283ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240062025-11-13T08:12:03.284ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
240072025-11-13T08:12:03.284ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
240082025-11-13T08:12:03.286ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240092025-11-13T08:12:03.286ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240102025-11-13T08:12:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240112025-11-13T08:12:03.288ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240122025-11-13T08:12:03.288ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240132025-11-13T08:12:03.289ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240142025-11-13T08:12:03.289ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240152025-11-13T08:12:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
240162025-11-13T08:12:03.292ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240172025-11-13T08:12:03.292ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240182025-11-13T08:12:03.293ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240192025-11-13T08:12:03.294ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240202025-11-13T08:12:03.295ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240212025-11-13T08:12:03.298ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
240222025-11-13T08:12:03.298ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240232025-11-13T08:12:03.298ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240242025-11-13T08:12:03.326ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240252025-11-13T08:12:03.327ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240262025-11-13T08:12:03.328ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240272025-11-13T08:12:03.328ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
240282025-11-13T08:12:03.328ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
240292025-11-13T08:12:03.331ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "210afea1-a9e6-43e2-833b-d1d6bd6d7e2b", "content-length": "372", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
240302025-11-13T08:12:03.331ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
240312025-11-13T08:12:03.331ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
240322025-11-13T08:12:03.332ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
240332025-11-13T08:12:03.332ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240342025-11-13T08:12:03.333ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240352025-11-13T08:12:03.334ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240362025-11-13T08:12:03.334ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240372025-11-13T08:12:03.334ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
240382025-11-13T08:12:03.334ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
240392025-11-13T08:12:03.335ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = 9b9e8fca-c443-4ab5-beec-8f9a1e390344 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
240402025-11-13T08:12:03.335ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240412025-11-13T08:12:03.335ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240422025-11-13T08:12:03.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240432025-11-13T08:12:03.336ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240442025-11-13T08:12:03.336ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240452025-11-13T08:12:03.338ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240462025-11-13T08:12:03.338ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240472025-11-13T08:12:03.339ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
240482025-11-13T08:12:03.339ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
240492025-11-13T08:12:03.339ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
240502025-11-13T08:12:03.341ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240512025-11-13T08:12:03.341ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240522025-11-13T08:12:03.341ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
240532025-11-13T08:12:03.341ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
240542025-11-13T08:12:03.342ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 724.002473ms iteration = 32 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
240552025-11-13T08:12:03.342ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240562025-11-13T08:12:03.342ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240572025-11-13T08:12:03.343ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
240582025-11-13T08:12:03.343ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
240592025-11-13T08:12:03.344ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240602025-11-13T08:12:03.344ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240612025-11-13T08:12:03.345ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240622025-11-13T08:12:03.347ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "906cb884-8b65-482f-9b0f-006102758b92", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
240632025-11-13T08:12:03.347ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
240642025-11-13T08:12:03.347ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
240652025-11-13T08:12:03.349ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
240662025-11-13T08:12:03.349ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
240672025-11-13T08:12:03.350ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
240682025-11-13T08:12:03.351ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240692025-11-13T08:12:03.351ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240702025-11-13T08:12:03.352ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
240712025-11-13T08:12:03.352ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240722025-11-13T08:12:03.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
240732025-11-13T08:12:03.352ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240742025-11-13T08:12:03.353ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240752025-11-13T08:12:03.353ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240762025-11-13T08:12:03.355ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240772025-11-13T08:12:03.355ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
240782025-11-13T08:12:03.355ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
240792025-11-13T08:12:03.355ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "ee6e85ad-09b4-4f49-9f98-17d62c8c7597", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
240802025-11-13T08:12:03.358ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 156.827122ms iteration = 41 status = {}
240812025-11-13T08:12:03.359ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
240822025-11-13T08:12:03.359ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240832025-11-13T08:12:03.359ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240842025-11-13T08:12:03.363ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240852025-11-13T08:12:03.364ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240862025-11-13T08:12:03.365ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240872025-11-13T08:12:03.365ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240882025-11-13T08:12:03.366ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240892025-11-13T08:12:03.367ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240902025-11-13T08:12:03.368ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240912025-11-13T08:12:03.369ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240922025-11-13T08:12:03.371ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240932025-11-13T08:12:03.371ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240942025-11-13T08:12:03.372ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240952025-11-13T08:12:03.372ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240962025-11-13T08:12:03.373ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
240972025-11-13T08:12:03.373ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
240982025-11-13T08:12:03.375ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
240992025-11-13T08:12:03.379ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
241002025-11-13T08:12:03.379ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241012025-11-13T08:12:03.380ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241022025-11-13T08:12:03.381ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241032025-11-13T08:12:03.381ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241042025-11-13T08:12:03.382ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241052025-11-13T08:12:03.382ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241062025-11-13T08:12:03.384ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241072025-11-13T08:12:03.389ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
241082025-11-13T08:12:03.389ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241092025-11-13T08:12:03.389ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241102025-11-13T08:12:03.394ZDEBGnexus (BackgroundTasks): resolved names for firewall rules background_task = blueprint_executor comment = initial blueprint from rack setup instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })], Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })]} vpc_interfaces = {}
241112025-11-13T08:12:03.394ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241122025-11-13T08:12:03.394ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241132025-11-13T08:12:03.395ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241142025-11-13T08:12:03.399ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
241152025-11-13T08:12:03.399ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241162025-11-13T08:12:03.399ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241172025-11-13T08:12:03.402ZDEBGnexus (BackgroundTasks): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = blueprint_executor comment = initial blueprint from rack setup
241182025-11-13T08:12:03.402ZDEBGnexus (BackgroundTasks): resolved firewall rules for sled agents background_task = blueprint_executor comment = initial blueprint from rack setup sled_agent_rules = [ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(53), last: L4Port(53) }]), filter_protocols: Some([Udp]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 840b1e6b-b274-4002-b2b1-ad8aed5df1e7, kind: Service { id: 25561cfb-881a-4ed6-9d55-6d9461030a3d }, name: Name("external-dns-25561cfb-881a-4ed6-9d55-6d9461030a3d"), ip: 172.30.1.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 150, 171])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: ecc1aa4c-5735-4d1d-a70e-47d4a9333691, kind: Service { id: bc3a50d3-9262-410c-8c88-068c994facb4 }, name: Name("external-dns-bc3a50d3-9262-410c-8c88-068c994facb4"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 232, 30])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: None, filter_protocols: Some([Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 3, code: Some(IcmpParamRange { first: 3, last: 4 }) })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 5, code: None })), Icmp(Some(VpcFirewallIcmpFilter { icmp_type: 11, code: None }))]), action: Allow, priority: VpcFirewallRulePriority(65534) }, ResolvedVpcFirewallRule { status: Enabled, direction: Inbound, targets: [NetworkInterface { id: 09011156-79af-478e-a9a6-f58417d803af, kind: Service { id: 79428d63-073a-4ba3-9687-ed29ef6cec3a }, name: Name("nexus-79428d63-073a-4ba3-9687-ed29ef6cec3a"), ip: 172.30.2.7, mac: MacAddr(MacAddr6([168, 64, 37, 255, 222, 16])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: 805451d3-c7d5-4457-abd3-64aec5d0fc02, kind: Service { id: 6564a4a0-ff58-4081-9860-74f24e488435 }, name: Name("nexus-6564a4a0-ff58-4081-9860-74f24e488435"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 207, 239])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }, NetworkInterface { id: f28f114f-d01d-4e8d-8078-ecd2c36a3bc0, kind: Service { id: bcb58129-79cc-406c-aa16-455a5cd9f031 }, name: Name("nexus-bcb58129-79cc-406c-aa16-455a5cd9f031"), ip: 172.30.2.5, mac: MacAddr(MacAddr6([168, 64, 37, 255, 225, 255])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], filter_hosts: None, filter_ports: Some([L4PortRange { first: L4Port(80), last: L4Port(80) }, L4PortRange { first: L4Port(443), last: L4Port(443) }]), filter_protocols: Some([Tcp]), action: Allow, priority: VpcFirewallRulePriority(65534) }]
241192025-11-13T08:12:03.402ZDEBGnexus (BackgroundTasks): resolved 3 rules for sleds background_task = blueprint_executor comment = initial blueprint from rack setup
241202025-11-13T08:12:03.461ZDEBGnexus (BackgroundTasks): resolved sleds for vpc oxide-services background_task = blueprint_executor comment = initial blueprint from rack setup vpc_to_sled = [Sled { identity: SledIdentity { id: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), time_created: 2025-11-13T07:59:02.661803Z, time_modified: 2025-11-13T07:59:02.661803Z }, time_deleted: None, rcgen: Generation(Generation(19)), rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, is_scrimlet: true, serial_number: "sock", part_number: "i86pc", revision: SqlU32(0), usable_hardware_threads: SqlU32(48), usable_physical_ram: ByteCount(ByteCount(102952026112)), reservoir_size: ByteCount(ByteCount(56094621696)), ip: fd00:1122:3344:101::1, port: SqlU16(12345), last_used_address: fd00:1122:3344:101::1:0, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(12348), cpu_family: AmdMilan }]
241212025-11-13T08:12:03.461ZDEBGnexus (BackgroundTasks): sending firewall rules to sled agents background_task = blueprint_executor comment = initial blueprint from rack setup
241222025-11-13T08:12:03.500ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241232025-11-13T08:12:03.500ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241242025-11-13T08:12:03.503ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241252025-11-13T08:12:03.503ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241262025-11-13T08:12:03.505ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241272025-11-13T08:12:03.508ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
241282025-11-13T08:12:03.508ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor body = Some(Body) comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
241292025-11-13T08:12:03.511ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "59cc9820-ff87-437a-8695-9d0c50c3d776", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
241302025-11-13T08:12:03.511ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
241312025-11-13T08:12:03.511ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
241322025-11-13T08:12:03.512ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfb49357-0898-441a-821a-50659f3e9a87", "content-length": "682", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
241332025-11-13T08:12:03.512ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
241342025-11-13T08:12:03.512ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
241352025-11-13T08:12:03.512ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "01d7bd2f-7353-4ecb-b08d-64d56a70ee07", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
241362025-11-13T08:12:03.514ZINFOnexus (BackgroundTasks): updated firewall rules on 1 sleds background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/networking/src/firewall_rules.rs:491
241372025-11-13T08:12:03.514ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = Dns step id = Ensure
241382025-11-13T08:12:03.514ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
241392025-11-13T08:12:03.514ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241402025-11-13T08:12:03.514ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241412025-11-13T08:12:03.515ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241422025-11-13T08:12:03.515ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241432025-11-13T08:12:03.516ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
241442025-11-13T08:12:03.516ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
241452025-11-13T08:12:03.516ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241462025-11-13T08:12:03.519ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
241472025-11-13T08:12:03.519ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
241482025-11-13T08:12:03.520ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
241492025-11-13T08:12:03.521ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
241502025-11-13T08:12:03.521ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
241512025-11-13T08:12:03.527ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 884.995788ms iteration = 27 status = {}
241522025-11-13T08:12:03.556ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal version = 1
241532025-11-13T08:12:03.556ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
241542025-11-13T08:12:03.556ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal
241552025-11-13T08:12:03.556ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241562025-11-13T08:12:03.556ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241572025-11-13T08:12:03.557ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241582025-11-13T08:12:03.560ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
241592025-11-13T08:12:03.560ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241602025-11-13T08:12:03.561ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241612025-11-13T08:12:03.595ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = internal
241622025-11-13T08:12:03.595ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
241632025-11-13T08:12:03.595ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241642025-11-13T08:12:03.595ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241652025-11-13T08:12:03.597ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241662025-11-13T08:12:03.600ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
241672025-11-13T08:12:03.600ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241682025-11-13T08:12:03.601ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241692025-11-13T08:12:03.603ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal dns_zone_id = d626fd93-1bb3-45b6-95f6-318b37acedd2 dns_zone_name = control-plane.oxide.internal found_so_far = 60 version = 1
241702025-11-13T08:12:03.603ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = internal nzones = 1 version = 1
241712025-11-13T08:12:03.603ZDEBGnexus (BackgroundTasks): reading DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
241722025-11-13T08:12:03.603ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241732025-11-13T08:12:03.603ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241742025-11-13T08:12:03.604ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241752025-11-13T08:12:03.604ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241762025-11-13T08:12:03.605ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241772025-11-13T08:12:03.610ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
241782025-11-13T08:12:03.646ZDEBGnexus (BackgroundTasks): found DNS version background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external version = 2
241792025-11-13T08:12:03.646ZDEBGnexus (BackgroundTasks): reading DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
241802025-11-13T08:12:03.646ZDEBGnexus (BackgroundTasks): reading DNS zones background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external
241812025-11-13T08:12:03.646ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241822025-11-13T08:12:03.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241832025-11-13T08:12:03.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241842025-11-13T08:12:03.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
241852025-11-13T08:12:03.652ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241862025-11-13T08:12:03.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241872025-11-13T08:12:03.691ZDEBGnexus (BackgroundTasks): found zones background_task = blueprint_executor comment = initial blueprint from rack setup count = 1 dns_group = external
241882025-11-13T08:12:03.691ZDEBGnexus (BackgroundTasks): listing DNS names for zone background_task = blueprint_executor batch_size = 100 comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 0 version = 2
241892025-11-13T08:12:03.691ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241902025-11-13T08:12:03.691ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241912025-11-13T08:12:03.693ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
241922025-11-13T08:12:03.696ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
241932025-11-13T08:12:03.696ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241942025-11-13T08:12:03.697ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241952025-11-13T08:12:03.699ZDEBGnexus (BackgroundTasks): found all DNS names for zone background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external dns_zone_id = 7bef1b14-06d8-492a-b4ce-2f250afc4fd5 dns_zone_name = oxide.test found_so_far = 4 version = 2
241962025-11-13T08:12:03.699ZDEBGnexus (BackgroundTasks): read DNS config background_task = blueprint_executor comment = initial blueprint from rack setup dns_group = external nzones = 1 version = 2
241972025-11-13T08:12:03.699ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
241982025-11-13T08:12:03.699ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
241992025-11-13T08:12:03.701ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242002025-11-13T08:12:03.703ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
242012025-11-13T08:12:03.703ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242022025-11-13T08:12:03.704ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242032025-11-13T08:12:03.705ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242042025-11-13T08:12:03.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242052025-11-13T08:12:03.706ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242062025-11-13T08:12:03.706ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242072025-11-13T08:12:03.708ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242082025-11-13T08:12:03.712ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = DnsConfig result = Ok(())
242092025-11-13T08:12:03.714ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns Internal comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
242102025-11-13T08:12:03.714ZINFOnexus (BackgroundTasks): no changes background_task = blueprint_executor blueprint_execution = dns External comment = initial blueprint from rack setup file = nexus/reconfigurator/execution/src/dns.rs:268
242112025-11-13T08:12:03.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = OmicronZones step id = Cleanup
242122025-11-13T08:12:03.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = OmicronZones step id = Cleanup
242132025-11-13T08:12:03.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = PhysicalDisks step id = Cleanup
242142025-11-13T08:12:03.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = Clickhouse step id = Ensure
242152025-11-13T08:12:03.714ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = Clickhouse step id = Ensure
242162025-11-13T08:12:03.744ZDEBGnexus (BackgroundTasks): client request admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor body = None comment = initial blueprint from rack setup method = PUT uri = http://[fd00:1122:3344:101::e]:8888/init
242172025-11-13T08:12:03.749ZDEBGnexus (BackgroundTasks): client response admin_url = http://[fd00:1122:3344:101::e]:8888 background_task = blueprint_executor comment = initial blueprint from rack setup result = Ok(Response { url: "http://[fd00:1122:3344:101::e]:8888/init", status: 204, headers: {"x-request-id": "8cf9ae9c-55c9-4dfa-944c-e132d2e5b185", "date": "Thu, 13 Nov 2025 08:12:03 GMT"} })
242182025-11-13T08:12:03.750ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = SupportBundles step id = Cleanup
242192025-11-13T08:12:03.750ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242202025-11-13T08:12:03.750ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242212025-11-13T08:12:03.753ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242222025-11-13T08:12:03.756ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
242232025-11-13T08:12:03.756ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242242025-11-13T08:12:03.756ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242252025-11-13T08:12:03.757ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242262025-11-13T08:12:03.757ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242272025-11-13T08:12:03.759ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242282025-11-13T08:12:03.762ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = BlueprintConfig result = Ok(())
242292025-11-13T08:12:03.800ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = OmicronZones step id = Cleanup
242302025-11-13T08:12:03.800ZINFOnexus (SagaQuiesceHandle): starting saga re-assignment pass file = nexus/types/src/quiesce.rs:390
242312025-11-13T08:12:03.800ZDEBGnexus (BackgroundTasks): re-assign sagas: found expunged Nexus instances with matching generation background_task = blueprint_executor comment = initial blueprint from rack setup nexus_zone_ids = []
242322025-11-13T08:12:03.800ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242332025-11-13T08:12:03.800ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242342025-11-13T08:12:03.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242352025-11-13T08:12:03.803ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Fleet result = Ok(())
242362025-11-13T08:12:03.803ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242372025-11-13T08:12:03.803ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242382025-11-13T08:12:03.838ZINFOnexus (BackgroundTasks): re-assigned sagas background_task = blueprint_executor comment = initial blueprint from rack setup count = 0 file = nexus/reconfigurator/execution/src/sagas.rs:62 nexus_zone_ids = []
242392025-11-13T08:12:03.838ZINFOnexus (SagaQuiesceHandle): saga re-assignment pass finished file = nexus/types/src/quiesce.rs:397 result = ReassignedAllAsOf(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 (blueprint), false)
242402025-11-13T08:12:03.838ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = Cockroach step id = Ensure
242412025-11-13T08:12:03.838ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_executor comment = initial blueprint from rack setup roles = RoleSet { roles: {} }
242422025-11-13T08:12:03.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_executor comment = initial blueprint from rack setup resource = Database result = Ok(())
242432025-11-13T08:12:03.854ZINFOnexus (BackgroundTasks): set cockroachdb setting background_task = blueprint_executor comment = initial blueprint from rack setup file = nexus/db-queries/src/db/datastore/cockroachdb_settings.rs:121 setting = cluster.preserve_downgrade_option value = 22.1
242442025-11-13T08:12:03.854ZDEBGnexus (UpdateEngine): start executing step background_task = blueprint_executor comment = initial blueprint from rack setup execution_id = e70101e3-80a7-4688-ac3f-64fda785c4c8 step component = MgsUpdates step id = Ensure
242452025-11-13T08:12:03.854ZDEBGnexus (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Dependency
242462025-11-13T08:12:03.854ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 41 reason = Dependency
242472025-11-13T08:12:03.854ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
242482025-11-13T08:12:03.854ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
242492025-11-13T08:12:03.855ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 1.247077336s iteration = 15 status = {"enabled":true,"event_report":{"Ok":{"last_seen":15,"progress_events":[],"root_execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","step_events":[{"data":{"components":[{"component":"ExternalNetworking","total_component_steps":1},{"component":"SledList","total_component_steps":1},{"component":"DeployNexusRecords","total_component_steps":1},{"component":"SledAgent","total_component_steps":1},{"component":"FirewallRules","total_component_steps":1},{"component":"Dns","total_component_steps":1},{"component":"OmicronZones","total_component_steps":3},{"component":"PhysicalDisks","total_component_steps":1},{"component":"Clickhouse","total_component_steps":2},{"component":"SupportBundles","total_component_steps":1},{"component":"Cockroach","total_component_steps":1},{"component":"MgsUpdates","total_component_steps":1}],"first_step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"kind":"execution_started","steps":[{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1}]},"event_index":0,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":3650,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":149303566,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"ExternalNetworking","component_index":0,"description":"Ensure external networking resources","id":"Ensure","index":0,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":149303515,"secs":0}},"event_index":1,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":149324806,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":100619388,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledList","component_index":0,"description":"Fetch sled list","id":"Fetch","index":1,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":100619338,"secs":0}},"event_index":2,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":249962484,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":86986840,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"DeployNexusRecords","component_index":0,"description":"Ensure db_metadata_nexus_state records exist","id":"Ensure","index":2,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":86986790,"secs":0}},"event_index":3,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":336962914,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":256999302,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"SledAgent","component_index":0,"description":"Deploy sled configs","id":"Ensure","index":3,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":256999252,"secs":0}},"event_index":4,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":593977957,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":312291188,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"FirewallRules","component_index":0,"description":"Plumb service firewall rules","id":"Ensure","index":4,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":312291128,"secs":0}},"event_index":5,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":906279706,"secs":0}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":199926930,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Dns","component_index":0,"description":"Deploy DNS records","id":"Ensure","index":5,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":199926890,"secs":0}},"event_index":6,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":106219557,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":19031,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":0,"description":"Cleanup expunged zones","id":"Cleanup","index":6,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":18991,"secs":0}},"event_index":7,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":106251188,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":2750,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":1,"description":"Decommission sleds","id":"Cleanup","index":7,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":2710,"secs":0}},"event_index":8,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":106266928,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":3270,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"PhysicalDisks","component_index":0,"description":"Decommission expunged disks","id":"Cleanup","index":8,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":3240,"secs":0}},"event_index":9,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":106272168,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":1750,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":0,"description":"Deploy clickhouse cluster nodes","id":"Ensure","index":9,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":1710,"secs":0}},"event_index":10,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":106280588,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":36071260,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Clickhouse","component_index":1,"description":"Deploy single-node clickhouse cluster","id":"Ensure","index":10,"total_component_steps":2},"metadata":null},"step_elapsed":{"nanos":36071209,"secs":0}},"event_index":11,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":142353088,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":49760638,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"outcome":{"kind":"success","message":"support bundle expunge report: SupportBundleExpungementReport { bundles_failed_missing_datasets: 0, bundles_deleted_missing_datasets: 0, bundles_failing_missing_nexus: 0, bundles_reassigned: 0 }","metadata":null},"step":{"info":{"component":"SupportBundles","component_index":0,"description":"Mark support bundles as failed if they rely on an expunged disk or sled","id":"Cleanup","index":11,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":49760608,"secs":0}},"event_index":12,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":192126206,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":37975127,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"OmicronZones","component_index":2,"description":"Reassign sagas","id":"Cleanup","index":12,"total_component_steps":3},"metadata":null},"step_elapsed":{"nanos":37975087,"secs":0}},"event_index":13,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":230113933,"secs":1}},{"data":{"attempt":1,"attempt_elapsed":{"nanos":16621977,"secs":0},"kind":"step_completed","next_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"outcome":{"kind":"success","message":null,"metadata":null},"step":{"info":{"component":"Cockroach","component_index":0,"description":"Ensure CockroachDB settings","id":"Ensure","index":13,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":16621927,"secs":0}},"event_index":14,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":246745250,"secs":1}},{"data":{"attempt_elapsed":{"nanos":4800,"secs":0},"kind":"execution_completed","last_attempt":1,"last_outcome":{"kind":"success","message":null,"metadata":null},"last_step":{"info":{"component":"MgsUpdates","component_index":0,"description":"Kick off MGS-managed updates","id":"Ensure","index":14,"total_component_steps":1},"metadata":null},"step_elapsed":{"nanos":4760,"secs":0}},"event_index":15,"execution_id":"77cc73f3-38f3-42b2-ae14-a22124787b0c","spec":"ReconfiguratorExecutionSpec","total_elapsed":{"nanos":246759240,"secs":1}}]}},"execution_error":null,"needs_saga_recovery":false,"target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200"}
242502025-11-13T08:12:03.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
242512025-11-13T08:12:03.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
242522025-11-13T08:12:03.856ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242532025-11-13T08:12:03.856ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242542025-11-13T08:12:03.859ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
242552025-11-13T08:12:03.859ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
242562025-11-13T08:12:03.859ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
242572025-11-13T08:12:03.859ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
242582025-11-13T08:12:03.859ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
242592025-11-13T08:12:03.860ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
242602025-11-13T08:12:03.874ZDEBGnexus (BackgroundTasks): inventory_prune_one: eligible for removal background_task = inventory_collection candidates = [(a7967a9a-1e1f-42c0-b592-9f2752cb200d, 0), (7d31dc28-379b-4924-a884-fecaf8907597, 0), (4b4e4760-c484-4e8b-bceb-8a994155167e, 0), (46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3, 0)] collection_id = a7967a9a-1e1f-42c0-b592-9f2752cb200d
242612025-11-13T08:12:03.874ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
242622025-11-13T08:12:03.874ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
242632025-11-13T08:12:03.895ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
242642025-11-13T08:12:03.895ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
242652025-11-13T08:12:03.897ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
242662025-11-13T08:12:03.899ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
242672025-11-13T08:12:03.899ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
242682025-11-13T08:12:03.899ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
242692025-11-13T08:12:03.932ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
242702025-11-13T08:12:03.932ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 77.754719ms iteration = 41 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
242712025-11-13T08:12:04.014ZWARNnexus (datastore): Retryable transaction failure file = nexus/db-queries/src/transaction_retry.rs:165 retry_after (ms) = 31 transaction = inventory_delete_collection
242722025-11-13T08:12:04.077ZINFOnexus (datastore): transaction completed attempts = 2 file = nexus/db-queries/src/transaction_retry.rs:121 transaction = inventory_delete_collection
242732025-11-13T08:12:04.077ZINFOnexus (BackgroundTasks): removed inventory collection background_task = inventory_collection collection_id = a7967a9a-1e1f-42c0-b592-9f2752cb200d file = nexus/db-queries/src/db/datastore/inventory.rs:2265 nboot_partitions = 0 ncabooses = 0 nclickhouse_keeper_membership = 0 ncockroach_status = 0 ncollections = 0 nconfig_reconcilers = 0 ndatasets = 0 nerrors = 0 nhost_phase1_active_slots = 0 nhost_phase1_flash_hashes = 0 ninternal_dns = 0 nlast_reconciliation_dataset_results = 0 nlast_reconciliation_disk_results = 0 nlast_reconciliation_orphaned_datasets = 0 nlast_reconciliation_zone_results = 0 nmupdate_override_non_boot = 0 nntp_timesync = 0 nnvme_disk_firmware = 0 nomicron_sled_config_datasets = 0 nomicron_sled_config_disks = 0 nomicron_sled_config_zone_nics = 0 nomicron_sled_config_zones = 0 nomicron_sled_configs = 0 nphysical_disks = 0 nrot_pages = 0 nrots = 0 nsled_agents = 0 nsps = 0 nzone_manifest_non_boot = 0 nzone_manifest_zones = 0 nzpools = 0
242742025-11-13T08:12:04.078ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
242752025-11-13T08:12:04.078ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
242762025-11-13T08:12:04.081ZDEBGnexus (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(7d31dc28-379b-4924-a884-fecaf8907597, 0), (4b4e4760-c484-4e8b-bceb-8a994155167e, 0), (46b5092e-e8f0-4da6-a5ab-6fa33e6e10d3, 0)]
242772025-11-13T08:12:04.081ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6732, tv_nsec: 433895785 } })
242782025-11-13T08:12:04.120ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _cockroach._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_cockroach._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("1edcd4a3-1ba4-4c9a-969e-a53b6d121af0.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("3e984927-121d-4c31-bcbc-7aff742b9962.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("6ebca908-3b9f-4f08-bb02-71c958d8ce45.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("a7965486-0a2e-4e4f-8d81-c8eb01477c9f.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("_cockroach._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 32221, target: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("faa4afc3-bf95-46d1-a155-a2f89ef9c47b.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::6)) }], valid_until: Instant { tv_sec: 6732, tv_nsec: 472677897 } })
242792025-11-13T08:12:04.121ZDEBGnexus (BackgroundTasks): begin collection background_task = inventory_collection
242802025-11-13T08:12:04.121ZDEBGnexus (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[fd00:1122:3344:101::2]:12225
242812025-11-13T08:12:04.121ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/ignition
242822025-11-13T08:12:04.121ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4560c0a-8131-49e0-b0b3-d1e40b252753", "content-length": "476", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242832025-11-13T08:12:04.121ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0
242842025-11-13T08:12:04.122ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ab2ba34-0939-4a95-a6ee-93652b0ccabd", "content-length": "734", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242852025-11-13T08:12:04.122ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0
242862025-11-13T08:12:04.122ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "75ff1203-b7bd-4e62-b05b-8491e7304f5d", "content-length": "111", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242872025-11-13T08:12:04.122ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1
242882025-11-13T08:12:04.122ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16a90bff-488e-4d63-b63d-69799a7724e1", "content-length": "111", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242892025-11-13T08:12:04.122ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0
242902025-11-13T08:12:04.123ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f84d12a-6cdb-49d8-aaa9-931fc6642347", "content-length": "163", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242912025-11-13T08:12:04.123ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1
242922025-11-13T08:12:04.123ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2aa66b87-111f-4e56-b360-db045bfe9649", "content-length": "163", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242932025-11-13T08:12:04.123ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0
242942025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff425d1-f571-4769-9306-8e40b6aa4831", "content-length": "166", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242952025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1
242962025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3ac6152-90a9-40c9-9568-026ec70c2014", "content-length": "166", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242972025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa
242982025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "556ae710-a64e-4f76-b6cb-375790033379", "content-length": "702", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
242992025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
243002025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dab40466-ed95-4568-9ad5-e79db36283ea", "content-length": "727", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243012025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
243022025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0f89180-c014-4973-8811-5f7d892bf3cc", "content-length": "729", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243032025-11-13T08:12:04.124ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa
243042025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1c31c01-dc62-4100-a8d2-ceea34da728a", "content-length": "728", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243052025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0
243062025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "829b9725-f7f5-4af3-b78d-a3cd7e9b79ba", "content-length": "722", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243072025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot
243082025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/active-slot", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d61118d-11ec-47b6-b0aa-ae42ea9df78b", "content-length": "10", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243092025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0
243102025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d555a507-9142-41a5-b93f-b28e0ab13eb4", "content-length": "144", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243112025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1
243122025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/host-boot-flash/hash/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f84d0593-49cd-47fa-823e-e39494e49a03", "content-length": "147", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243132025-11-13T08:12:04.125ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0
243142025-11-13T08:12:04.126ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "548f54d5-3c92-4e9e-ad2a-77554f66988b", "content-length": "109", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243152025-11-13T08:12:04.126ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1
243162025-11-13T08:12:04.126ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3400f72b-051e-484d-a1cd-3762bbbac8d5", "content-length": "109", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243172025-11-13T08:12:04.126ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0
243182025-11-13T08:12:04.126ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "80cd730b-9798-4a99-abb6-7664354c27ec", "content-length": "163", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243192025-11-13T08:12:04.126ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1
243202025-11-13T08:12:04.127ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b44a24a-4338-4690-8639-89313c1654e6", "content-length": "163", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243212025-11-13T08:12:04.127ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0
243222025-11-13T08:12:04.127ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff18b6d0-6ac5-4f02-bd58-a7a3316fc1b1", "content-length": "166", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243232025-11-13T08:12:04.127ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1
243242025-11-13T08:12:04.127ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ce4eb9b-5398-42ce-b52f-f5206788e10a", "content-length": "166", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243252025-11-13T08:12:04.127ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa
243262025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a965d89-7e46-458a-9ac3-c886953c7e85", "content-length": "702", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243272025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
243282025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb7a0aa7-d3d8-447d-9a18-b71fd36138df", "content-length": "727", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243292025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
243302025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60333b38-ab52-4fe7-a0c8-d341b524cd5a", "content-length": "729", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243312025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa
243322025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1c39743-962d-4947-a7de-a9f0e239727c", "content-length": "728", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243332025-11-13T08:12:04.128ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
243342025-11-13T08:12:04.129ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
243352025-11-13T08:12:04.130ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243362025-11-13T08:12:04.133ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
243372025-11-13T08:12:04.133ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
243382025-11-13T08:12:04.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
243392025-11-13T08:12:04.134ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243402025-11-13T08:12:04.136ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Fleet result = Ok(())
243412025-11-13T08:12:04.136ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
243422025-11-13T08:12:04.136ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
243432025-11-13T08:12:04.169ZDEBGnexus (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1]:12345
243442025-11-13T08:12:04.169ZDEBGnexus (BackgroundTasks): client request SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/inventory
243452025-11-13T08:12:04.220ZDEBGnexus (BackgroundTasks): client response SledAgent = http://[fd00:1122:3344:101::1]:12345 background_task = inventory_collection result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c96a631b-6867-4732-b732-d824c2a291fb", "content-length": "95408", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243462025-11-13T08:12:04.223ZDEBGnexus (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
243472025-11-13T08:12:04.223ZDEBGnexus (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
243482025-11-13T08:12:04.223ZDEBGnexus (BackgroundTasks): begin collection from CockroachDB nodes background_task = inventory_collection
243492025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/node/id
243502025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/node/id
243512025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/node/id
243522025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/node/id
243532025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/node/id
243542025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7f0872c-d169-4095-8751-c18ce4e83fd7", "content-length": "64", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243552025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::3]:32222/proxy/status/vars
243562025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "04abeb92-b8e4-416d-bbea-bb28a75ad287", "content-length": "64", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243572025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "491e4269-4f25-4e52-90d3-e4fa69708685", "content-length": "64", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243582025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "5db6669e-3083-4d7b-a4f7-ed1e34946951", "content-length": "64", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243592025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::7]:32222/proxy/status/vars
243602025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::5]:32222/proxy/status/vars
243612025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/node/id", status: 200, headers: {"content-type": "application/json", "x-request-id": "000f8e57-8599-4631-ae8b-efee5d6ec84d", "content-length": "64", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243622025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::6]:32222/proxy/status/vars
243632025-11-13T08:12:04.223ZDEBGnexus (CockroachClusterAdminClient): client request body = None method = GET uri = http://[fd00:1122:3344:101::4]:32222/proxy/status/vars
243642025-11-13T08:12:04.261ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::4]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "776be0a6-4367-40b5-aa84-b095fd81181c", "content-length": "416539", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243652025-11-13T08:12:04.261ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::5]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "895cb0da-4ec1-42bb-a05f-3f3997da9815", "content-length": "411761", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243662025-11-13T08:12:04.261ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::7]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6b594d0-1575-4560-9142-2b7675d38a5f", "content-length": "431381", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243672025-11-13T08:12:04.262ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::6]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c31009e-07d8-4bef-ab2e-86c4311c5c09", "content-length": "410973", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243682025-11-13T08:12:04.268ZDEBGnexus (CockroachClusterAdminClient): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::3]:32222/proxy/status/vars", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e706335-8e05-4249-ac9d-78bf0257ed4c", "content-length": "411730", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243692025-11-13T08:12:04.270ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 5
243702025-11-13T08:12:04.270ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 2
243712025-11-13T08:12:04.270ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 1
243722025-11-13T08:12:04.270ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 3
243732025-11-13T08:12:04.270ZDEBGnexus (CockroachClusterAdminClient): Successfully fetched metrics from CockroachDB node node_id = 4
243742025-11-13T08:12:04.301ZDEBGnexus (BackgroundTasks): begin collection from NTP admin (timesync) background_task = inventory_collection sled_agent_url = http://[fd00:1122:3344:101::1b]:10123 zone_id = 83a0d63c-a7a2-4e4a-aeca-da01f3d52209 (omicron_zone)
243752025-11-13T08:12:04.301ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None method = GET ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 uri = http://[fd00:1122:3344:101::1b]:10123/timesync
243762025-11-13T08:12:04.304ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection ntp_admin_url = http://[fd00:1122:3344:101::1b]:10123 result = Ok(Response { url: "http://[fd00:1122:3344:101::1b]:10123/timesync", status: 200, headers: {"content-type": "application/json", "x-request-id": "acc60a9b-bcae-4cf5-b43e-d7a0dd6e01fd", "content-length": "124", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243772025-11-13T08:12:04.305ZDEBGnexus (BackgroundTasks): begin collection from internal DNS servers background_task = inventory_collection
243782025-11-13T08:12:04.394ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
243792025-11-13T08:12:04.394ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:2::1]:5353 method = GET uri = http://[fd00:1122:3344:2::1]:5353/config
243802025-11-13T08:12:04.395ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:2::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:2::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccf9d3e9-7da8-45ed-b159-996c8bb6613a", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243812025-11-13T08:12:04.395ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 0b826b69-7e7f-4662-88f8-01d591c1a724 (omicron_zone)
243822025-11-13T08:12:04.397ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
243832025-11-13T08:12:04.397ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:3::1]:5353 method = GET uri = http://[fd00:1122:3344:3::1]:5353/config
243842025-11-13T08:12:04.397ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:3::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:3::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "20045c9b-3746-4822-be90-ce62f421395c", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243852025-11-13T08:12:04.397ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 353e9780-feb1-419a-a764-07869c5904de (omicron_zone)
243862025-11-13T08:12:04.398ZDEBGnexus (BackgroundTasks): begin collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
243872025-11-13T08:12:04.398ZDEBGnexus (BackgroundTasks): client request background_task = inventory_collection body = None internal_dns_url = http://[fd00:1122:3344:1::1]:5353 method = GET uri = http://[fd00:1122:3344:1::1]:5353/config
243882025-11-13T08:12:04.399ZDEBGnexus (BackgroundTasks): client response background_task = inventory_collection internal_dns_url = http://[fd00:1122:3344:1::1]:5353 result = Ok(Response { url: "http://[fd00:1122:3344:1::1]:5353/config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a13d59b8-0bb7-49e8-b28b-df9fdf25f7cd", "content-length": "9552", "date": "Thu, 13 Nov 2025 08:12:04 GMT"} })
243892025-11-13T08:12:04.399ZDEBGnexus (BackgroundTasks): finished collection from DNS server background_task = inventory_collection zone_id = 55b55228-bf82-4bcb-b680-ded78e45bf10 (omicron_zone)
243902025-11-13T08:12:04.399ZDEBGnexus (BackgroundTasks): finished collection from internal DNS servers background_task = inventory_collection
243912025-11-13T08:12:04.399ZDEBGnexus (BackgroundTasks): finished collection background_task = inventory_collection
243922025-11-13T08:12:04.401ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
243932025-11-13T08:12:04.402ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
243942025-11-13T08:12:04.403ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
243952025-11-13T08:12:04.406ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Inventory result = Ok(())
243962025-11-13T08:12:04.406ZDEBGnexus (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
243972025-11-13T08:12:04.406ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_collection resource = Database result = Ok(())
243982025-11-13T08:12:04.580ZINFOnexus (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44 file = nexus/db-queries/src/db/datastore/inventory.rs:1613
243992025-11-13T08:12:04.580ZDEBGnexus (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44 time_started = 2025-11-13 08:12:04.121268 UTC
244002025-11-13T08:12:04.580ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 69 reason = Dependency
244012025-11-13T08:12:04.580ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
244022025-11-13T08:12:04.580ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 725.983553ms iteration = 16 status = {"collection_id":"4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44","time_done":"2025-11-13 08:12:04.399970 UTC","time_started":"2025-11-13 08:12:04.121268 UTC"}
244032025-11-13T08:12:04.581ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
244042025-11-13T08:12:04.582ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244052025-11-13T08:12:04.586ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
244062025-11-13T08:12:04.586ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
244072025-11-13T08:12:04.586ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
244082025-11-13T08:12:04.588ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
244092025-11-13T08:12:04.588ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
244102025-11-13T08:12:04.635ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 54.794097ms iteration = 69 status = {"Loaded":{"collection_id":"4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44","time_loaded":"2025-11-13T08:12:04.580804732Z","time_started":"2025-11-13T08:12:04.121268Z"}}
244112025-11-13T08:12:04.635ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 20 reason = Dependency
244122025-11-13T08:12:04.635ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 44 reason = Dependency
244132025-11-13T08:12:04.635ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_planner iteration = 33 reason = Dependency
244142025-11-13T08:12:04.635ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
244152025-11-13T08:12:04.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244162025-11-13T08:12:04.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244172025-11-13T08:12:04.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
244182025-11-13T08:12:04.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244192025-11-13T08:12:04.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244202025-11-13T08:12:04.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244212025-11-13T08:12:04.637ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244222025-11-13T08:12:04.637ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244232025-11-13T08:12:04.640ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
244242025-11-13T08:12:04.640ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244252025-11-13T08:12:04.640ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244262025-11-13T08:12:04.641ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
244272025-11-13T08:12:04.641ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244282025-11-13T08:12:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
244292025-11-13T08:12:04.641ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
244302025-11-13T08:12:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244312025-11-13T08:12:04.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
244322025-11-13T08:12:04.642ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244332025-11-13T08:12:04.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244342025-11-13T08:12:04.644ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
244352025-11-13T08:12:04.644ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
244362025-11-13T08:12:04.644ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244372025-11-13T08:12:04.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
244382025-11-13T08:12:04.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244392025-11-13T08:12:04.645ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
244402025-11-13T08:12:04.645ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244412025-11-13T08:12:04.645ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244422025-11-13T08:12:04.645ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244432025-11-13T08:12:04.647ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244442025-11-13T08:12:04.647ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244452025-11-13T08:12:04.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
244462025-11-13T08:12:04.648ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
244472025-11-13T08:12:04.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
244482025-11-13T08:12:04.649ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244492025-11-13T08:12:04.651ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
244502025-11-13T08:12:04.651ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244512025-11-13T08:12:04.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244522025-11-13T08:12:04.653ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244532025-11-13T08:12:04.658ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
244542025-11-13T08:12:04.658ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244552025-11-13T08:12:04.658ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244562025-11-13T08:12:04.660ZINFOnexus (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/crucible_dataset.rs:110 num_already_exist = 9 num_inserted = 0 num_not_in_inventory = 0
244572025-11-13T08:12:04.660ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244582025-11-13T08:12:04.660ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244592025-11-13T08:12:04.665ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244602025-11-13T08:12:04.671ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
244612025-11-13T08:12:04.671ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244622025-11-13T08:12:04.671ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244632025-11-13T08:12:04.673ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244642025-11-13T08:12:04.676ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
244652025-11-13T08:12:04.676ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
244662025-11-13T08:12:04.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
244672025-11-13T08:12:04.679ZINFOnexus (BackgroundTasks): all LocalStorage datasets reconciled background_task = blueprint_rendezvous file = nexus/reconfigurator/rendezvous/src/local_storage_dataset.rs:145 num_already_exist = 9 num_already_tombstoned = 0 num_inserted = 0 num_not_in_inventory = 0 num_tombstoned = 0
244682025-11-13T08:12:04.679ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 43.844052ms iteration = 20 status = {"blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","inventory_collection_id":"4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44","stats":{"crucible_dataset":{"num_already_exist":9,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0},"local_storage_dataset":{"num_already_exist":9,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
244692025-11-13T08:12:04.697ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 61.635802ms iteration = 44 status = {"physical_disks_added":0}
244702025-11-13T08:12:04.729ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244712025-11-13T08:12:04.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244722025-11-13T08:12:04.730ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244732025-11-13T08:12:04.732ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
244742025-11-13T08:12:04.732ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244752025-11-13T08:12:04.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244762025-11-13T08:12:04.734ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244772025-11-13T08:12:04.736ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
244782025-11-13T08:12:04.736ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244792025-11-13T08:12:04.736ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244802025-11-13T08:12:04.771ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244812025-11-13T08:12:04.772ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244822025-11-13T08:12:04.773ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244832025-11-13T08:12:04.774ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244842025-11-13T08:12:04.775ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244852025-11-13T08:12:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
244862025-11-13T08:12:04.778ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244872025-11-13T08:12:04.778ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244882025-11-13T08:12:04.780ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244892025-11-13T08:12:04.780ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244902025-11-13T08:12:04.781ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244912025-11-13T08:12:04.785ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
244922025-11-13T08:12:04.785ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244932025-11-13T08:12:04.785ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244942025-11-13T08:12:04.786ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
244952025-11-13T08:12:04.789ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
244962025-11-13T08:12:04.789ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244972025-11-13T08:12:04.789ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
244982025-11-13T08:12:04.791ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
244992025-11-13T08:12:04.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245002025-11-13T08:12:04.792ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245012025-11-13T08:12:04.795ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 94c21402-bf93-4796-877c-53d332e6d2ad, lookup_type: ByName("oxide-service-pool-v6") } result = Ok(())
245022025-11-13T08:12:04.795ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245032025-11-13T08:12:04.795ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245042025-11-13T08:12:04.797ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245052025-11-13T08:12:04.798ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245062025-11-13T08:12:04.799ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245072025-11-13T08:12:04.802ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
245082025-11-13T08:12:04.802ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245092025-11-13T08:12:04.802ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245102025-11-13T08:12:04.805ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245112025-11-13T08:12:04.806ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245122025-11-13T08:12:04.808ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245132025-11-13T08:12:04.812ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
245142025-11-13T08:12:04.812ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245152025-11-13T08:12:04.812ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245162025-11-13T08:12:04.829ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245172025-11-13T08:12:04.829ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245182025-11-13T08:12:04.831ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245192025-11-13T08:12:04.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245202025-11-13T08:12:04.832ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245212025-11-13T08:12:04.835ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
245222025-11-13T08:12:04.835ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245232025-11-13T08:12:04.835ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245242025-11-13T08:12:04.837ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245252025-11-13T08:12:04.840ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
245262025-11-13T08:12:04.840ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245272025-11-13T08:12:04.840ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245282025-11-13T08:12:04.841ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245292025-11-13T08:12:04.841ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245302025-11-13T08:12:04.843ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245312025-11-13T08:12:04.843ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245322025-11-13T08:12:04.845ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245332025-11-13T08:12:04.848ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
245342025-11-13T08:12:04.848ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245352025-11-13T08:12:04.848ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245362025-11-13T08:12:04.849ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245372025-11-13T08:12:04.852ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = IpPool { parent: Fleet, key: 19eb0509-0e22-4b5d-ac3e-a96724ba46d9, lookup_type: ByName("oxide-service-pool-v4") } result = Ok(())
245382025-11-13T08:12:04.852ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245392025-11-13T08:12:04.852ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245402025-11-13T08:12:04.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245412025-11-13T08:12:04.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245422025-11-13T08:12:04.855ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245432025-11-13T08:12:04.855ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245442025-11-13T08:12:04.857ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245452025-11-13T08:12:04.860ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
245462025-11-13T08:12:04.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245472025-11-13T08:12:04.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245482025-11-13T08:12:04.861ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245492025-11-13T08:12:04.861ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245502025-11-13T08:12:04.863ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245512025-11-13T08:12:04.866ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = DnsConfig result = Ok(())
245522025-11-13T08:12:04.867ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245532025-11-13T08:12:04.867ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245542025-11-13T08:12:04.869ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245552025-11-13T08:12:04.869ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245562025-11-13T08:12:04.871ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245572025-11-13T08:12:04.873ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
245582025-11-13T08:12:04.873ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245592025-11-13T08:12:04.873ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245602025-11-13T08:12:04.875ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245612025-11-13T08:12:04.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245622025-11-13T08:12:04.876ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245632025-11-13T08:12:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = TargetReleaseConfig result = Ok(())
245642025-11-13T08:12:04.879ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245652025-11-13T08:12:04.879ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245662025-11-13T08:12:04.880ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245672025-11-13T08:12:04.880ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245682025-11-13T08:12:04.881ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245692025-11-13T08:12:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
245702025-11-13T08:12:04.884ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245712025-11-13T08:12:04.884ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245722025-11-13T08:12:04.886ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245732025-11-13T08:12:04.887ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245742025-11-13T08:12:04.888ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245752025-11-13T08:12:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Fleet result = Ok(())
245762025-11-13T08:12:04.890ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245772025-11-13T08:12:04.890ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245782025-11-13T08:12:04.924ZDEBGnexus (BackgroundTasks): no mupdate override action taken, current value left unchanged background_task = blueprint_planner mupdate_override = None phase = do_plan_mupdate_override sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
245792025-11-13T08:12:04.924ZINFOnexus (BackgroundTasks): skipping noop image source check for all sleds background_task = blueprint_planner file = nexus/reconfigurator/planning/src/planner/image_source.rs:149 reason = no target release is currently set
245802025-11-13T08:12:04.924ZDEBGnexus (BlueprintBuilder): sled unchanged in new blueprint background_task = blueprint_planner blueprint_id = d25fd83f-9dbc-4e85-8025-4adec87dd574 parent_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 sled_id = 370ec2da-8f7f-4861-9b77-369c2bb1c86d
245812025-11-13T08:12:04.924ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245822025-11-13T08:12:04.925ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245832025-11-13T08:12:04.926ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245842025-11-13T08:12:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = BlueprintConfig result = Ok(())
245852025-11-13T08:12:04.929ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_planner roles = RoleSet { roles: {} }
245862025-11-13T08:12:04.929ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_planner resource = Database result = Ok(())
245872025-11-13T08:12:04.932ZDEBGnexus (BackgroundTasks): blueprint count under limit, proceeding with planning background_task = blueprint_planner count = 2 limit = 5000 usage_percent = 0
245882025-11-13T08:12:04.932ZINFOnexus (BackgroundTasks): blueprint unchanged from current target background_task = blueprint_planner file = nexus/src/app/background/tasks/blueprint_planner.rs:228 parent_blueprint_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200
245892025-11-13T08:12:04.932ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_planner elapsed = 296.689523ms iteration = 33 status = {"Unchanged":{"blueprint_count":2,"limit":5000,"parent_blueprint_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","report":{"add":{"add_update_blocked_reasons":["sleds have deployment units with image sources not set to Artifact:\\n- sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d: 28 zones\\n"],"add_zones_with_mupdate_override":false,"discretionary_zones_placed":{},"out_of_eligible_sleds":{"boundary_ntp":{"placed":0,"wanted_to_place":1}},"sleds_getting_ntp_and_discretionary_zones":[],"sleds_missing_crucible_zone":{},"sleds_missing_ntp_zone":[],"sleds_waiting_for_ntp_zone":[],"sleds_without_ntp_zones_in_inventory":[],"sleds_without_zpools_for_ntp_zones":[],"sufficient_zones_exist":{"clickhouse":{"num_existing":1,"target_count":1},"clickhouse_keeper":{"num_existing":0,"target_count":0},"clickhouse_server":{"num_existing":0,"target_count":0},"cockroach_db":{"num_existing":5,"target_count":5},"crucible_pantry":{"num_existing":3,"target_count":3},"external_dns":{"num_existing":2,"target_count":2},"internal_dns":{"num_existing":3,"target_count":3},"nexus":{"num_existing":3,"target_count":3},"oximeter":{"num_existing":1,"target_count":1}},"target_release_generation_is_one":true,"waiting_on":null},"cockroachdb_settings":{"preserve_downgrade":{"action":"set","data":"V22_1"}},"decommission":{"zombie_sleds":[]},"expunge":{"orphan_disks":{}},"mgs_updates":{"blocked_mgs_updates":[],"pending_mgs_updates":{"by_baseboard":[]}},"nexus_generation_bump":{"component":"nothing_to_report"},"noop_image_source":{"converted":{},"no_target_release":false,"skipped_sled_host_phase_2":{},"skipped_sled_zones":{},"skipped_zones":{},"sled_host_phase_2_both_already_artifact":[],"sled_zones_all_already_artifact":[]},"planner_config":{"add_zones_with_mupdate_override":false},"zone_updates":{"expunged_zones":{},"out_of_date_zones":{},"unsafe_zones":{},"updated_zones":{},"waiting_on":{"type":"zone_add_blockers"},"waiting_zones":{}}}}}
245902025-11-13T08:12:05.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 323 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 553ebc0a-c161-400b-9970-3ceadc7d3a95 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
245912025-11-13T08:12:07.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 158 reason = Timeout
245922025-11-13T08:12:07.627ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
245932025-11-13T08:12:07.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
245942025-11-13T08:12:07.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
245952025-11-13T08:12:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
245962025-11-13T08:12:07.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
245972025-11-13T08:12:07.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
245982025-11-13T08:12:07.635ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
245992025-11-13T08:12:07.635ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.124439ms iteration = 158 status = {"config_updated":false}
246002025-11-13T08:12:12.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 80 reason = Timeout
246012025-11-13T08:12:12.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 159 reason = Timeout
246022025-11-13T08:12:12.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246032025-11-13T08:12:12.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246042025-11-13T08:12:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246052025-11-13T08:12:12.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246062025-11-13T08:12:12.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246072025-11-13T08:12:12.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246082025-11-13T08:12:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
246092025-11-13T08:12:12.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246102025-11-13T08:12:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246112025-11-13T08:12:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
246122025-11-13T08:12:12.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246132025-11-13T08:12:12.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246142025-11-13T08:12:12.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246152025-11-13T08:12:12.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
246162025-11-13T08:12:12.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.598ms iteration = 159 status = {"config_updated":false}
246172025-11-13T08:12:12.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246182025-11-13T08:12:12.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246192025-11-13T08:12:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
246202025-11-13T08:12:12.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246212025-11-13T08:12:12.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246222025-11-13T08:12:12.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 80 reason = Timeout
246232025-11-13T08:12:12.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
246242025-11-13T08:12:12.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
246252025-11-13T08:12:12.644ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
246262025-11-13T08:12:12.644ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
246272025-11-13T08:12:12.654ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
246282025-11-13T08:12:12.654ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.873158ms iteration = 80 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
246292025-11-13T08:12:12.680ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 37.356324ms iteration = 80 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
246302025-11-13T08:12:15.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 317 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 8e29e052-a1c4-405e-847f-98caedc17a2d response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
246312025-11-13T08:12:17.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 70 reason = Timeout
246322025-11-13T08:12:17.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 160 reason = Timeout
246332025-11-13T08:12:17.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
246342025-11-13T08:12:17.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246352025-11-13T08:12:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
246362025-11-13T08:12:17.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246372025-11-13T08:12:17.628ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246382025-11-13T08:12:17.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246392025-11-13T08:12:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
246402025-11-13T08:12:17.632ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
246412025-11-13T08:12:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
246422025-11-13T08:12:17.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246432025-11-13T08:12:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
246442025-11-13T08:12:17.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246452025-11-13T08:12:17.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
246462025-11-13T08:12:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.220822ms iteration = 160 status = {"config_updated":false}
246472025-11-13T08:12:17.634ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44 original_time_started = 2025-11-13 08:12:04.121268 UTC
246482025-11-13T08:12:17.634ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 8.361294ms iteration = 70 status = {"Loaded":{"collection_id":"4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44","time_loaded":"2025-11-13T08:12:17.626121673Z","time_started":"2025-11-13T08:12:04.121268Z"}}
246492025-11-13T08:12:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 54 reason = Timeout
246502025-11-13T08:12:17.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 54 reason = Timeout
246512025-11-13T08:12:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
246522025-11-13T08:12:17.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
246532025-11-13T08:12:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
246542025-11-13T08:12:17.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
246552025-11-13T08:12:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
246562025-11-13T08:12:17.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
246572025-11-13T08:12:17.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246582025-11-13T08:12:17.650ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246592025-11-13T08:12:17.652ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
246602025-11-13T08:12:17.652ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
246612025-11-13T08:12:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
246622025-11-13T08:12:17.652ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
246632025-11-13T08:12:17.654ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 5.832365ms iteration = 54 status = "NoSitrep"
246642025-11-13T08:12:17.654ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
246652025-11-13T08:12:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
246662025-11-13T08:12:17.655ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
246672025-11-13T08:12:17.655ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
246682025-11-13T08:12:17.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246692025-11-13T08:12:17.658ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
246702025-11-13T08:12:17.658ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
246712025-11-13T08:12:17.659ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 10.512407ms iteration = 54 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
246722025-11-13T08:12:22.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 81 reason = Timeout
246732025-11-13T08:12:22.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 161 reason = Timeout
246742025-11-13T08:12:22.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246752025-11-13T08:12:22.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246762025-11-13T08:12:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246772025-11-13T08:12:22.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246782025-11-13T08:12:22.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246792025-11-13T08:12:22.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246802025-11-13T08:12:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
246812025-11-13T08:12:22.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
246822025-11-13T08:12:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
246832025-11-13T08:12:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
246842025-11-13T08:12:22.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246852025-11-13T08:12:22.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246862025-11-13T08:12:22.634ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246872025-11-13T08:12:22.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
246882025-11-13T08:12:22.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 7.888375ms iteration = 161 status = {"config_updated":false}
246892025-11-13T08:12:22.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246902025-11-13T08:12:22.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
246912025-11-13T08:12:22.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
246922025-11-13T08:12:22.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
246932025-11-13T08:12:22.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
246942025-11-13T08:12:22.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 81 reason = Timeout
246952025-11-13T08:12:22.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
246962025-11-13T08:12:22.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
246972025-11-13T08:12:22.650ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
246982025-11-13T08:12:22.650ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 24.454171ms iteration = 81 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
246992025-11-13T08:12:22.672ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
247002025-11-13T08:12:22.673ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
247012025-11-13T08:12:22.707ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 64.946247ms iteration = 81 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
247022025-11-13T08:12:25.480ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 276 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 72a8f125-27f1-48e9-9b06-0584623f6d7a response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
247032025-11-13T08:12:27.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 162 reason = Timeout
247042025-11-13T08:12:27.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247052025-11-13T08:12:27.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247062025-11-13T08:12:27.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247072025-11-13T08:12:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
247082025-11-13T08:12:27.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247092025-11-13T08:12:27.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247102025-11-13T08:12:27.632ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
247112025-11-13T08:12:27.632ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.649341ms iteration = 162 status = {"config_updated":false}
247122025-11-13T08:12:32.607ZDEBGnexus (BackgroundTasks): activating background_task = phantom_disks iteration = 28 reason = Timeout
247132025-11-13T08:12:32.607ZDEBGnexus (BackgroundTasks): activating background_task = nat_garbage_collector iteration = 28 reason = Timeout
247142025-11-13T08:12:32.607ZDEBGnexus (BackgroundTasks): activating background_task = bfd_manager iteration = 28 reason = Timeout
247152025-11-13T08:12:32.607ZDEBGnexus (BackgroundTasks): roles background_task = nat_garbage_collector roles = RoleSet { roles: {} }
247162025-11-13T08:12:32.607ZDEBGnexus (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
247172025-11-13T08:12:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = nat_garbage_collector resource = Database result = Ok(())
247182025-11-13T08:12:32.608ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = bfd_manager resource = Database result = Ok(())
247192025-11-13T08:12:32.611ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6760, tv_nsec: 963125328 } })
247202025-11-13T08:12:32.611ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6760, tv_nsec: 963171159 } })
247212025-11-13T08:12:32.613ZDEBGnexus (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 6.026709ms iteration = 28 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
247222025-11-13T08:12:32.614ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager file = nexus/src/app/mod.rs:1332
247232025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): activating background_task = support_bundle_collector iteration = 42 reason = Timeout
247242025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
247252025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): activating background_task = inventory_loader iteration = 71 reason = Timeout
247262025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 82 reason = Timeout
247272025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
247282025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
247292025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 45 reason = Timeout
247302025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 163 reason = Timeout
247312025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247322025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
247332025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
247342025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247352025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
247362025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
247372025-11-13T08:12:32.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
247382025-11-13T08:12:32.628ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247392025-11-13T08:12:32.628ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247402025-11-13T08:12:32.629ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247412025-11-13T08:12:32.629ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247422025-11-13T08:12:32.629ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247432025-11-13T08:12:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
247442025-11-13T08:12:32.632ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
247452025-11-13T08:12:32.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
247462025-11-13T08:12:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
247472025-11-13T08:12:32.634ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
247482025-11-13T08:12:32.634ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
247492025-11-13T08:12:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
247502025-11-13T08:12:32.635ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
247512025-11-13T08:12:32.635ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
247522025-11-13T08:12:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
247532025-11-13T08:12:32.635ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
247542025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Inventory result = Ok(())
247552025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
247562025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
247572025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): roles background_task = inventory_loader roles = RoleSet { roles: {} }
247582025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
247592025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 10.383214ms iteration = 163 status = {"config_updated":false}
247602025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = inventory_loader resource = Database result = Ok(())
247612025-11-13T08:12:32.636ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
247622025-11-13T08:12:32.637ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247632025-11-13T08:12:32.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
247642025-11-13T08:12:32.638ZDEBGnexus (BackgroundTasks): latest inventory collection is unchanged background_task = inventory_loader original_id = 4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44 original_time_started = 2025-11-13 08:12:04.121268 UTC
247652025-11-13T08:12:32.638ZDEBGnexus (BackgroundTasks): activation complete background_task = inventory_loader elapsed = 12.619798ms iteration = 71 status = {"Loaded":{"collection_id":"4cb4dd54-55b5-48ed-b5c5-972c2f4b6b44","time_loaded":"2025-11-13T08:12:32.626302533Z","time_started":"2025-11-13T08:12:04.121268Z"}}
247662025-11-13T08:12:32.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
247672025-11-13T08:12:32.640ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
247682025-11-13T08:12:32.640ZDEBGnexus (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
247692025-11-13T08:12:32.641ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
247702025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): activating background_task = instance_watcher iteration = 28 reason = Timeout
247712025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
247722025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): activating background_task = v2p_manager iteration = 28 reason = Timeout
247732025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 28 reason = Timeout
247742025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
247752025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
247762025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
247772025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
247782025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
247792025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 82 reason = Timeout
247802025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
247812025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): activating background_task = vpc_route_manager iteration = 42 reason = Timeout
247822025-11-13T08:12:32.642ZINFOnexus (BackgroundTasks): VPC route manager running background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:60
247832025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
247842025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 28 reason = Timeout
247852025-11-13T08:12:32.642ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
247862025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
247872025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
247882025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
247892025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247902025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement iteration = 28 reason = Timeout
247912025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
247922025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): activating background_task = instance_updater iteration = 28 reason = Timeout
247932025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
247942025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
247952025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
247962025-11-13T08:12:32.643ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
247972025-11-13T08:12:32.647ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247982025-11-13T08:12:32.647ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
247992025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
248002025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 28 reason = Timeout
248012025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
248022025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 28 reason = Timeout
248032025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
248042025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 28 reason = Timeout
248052025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
248062025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 28 reason = Timeout
248072025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_loader iteration = 55 reason = Timeout
248082025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
248092025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
248102025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 28 reason = Timeout
248112025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
248122025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = sp_ereport_ingester iteration = 28 reason = Timeout
248132025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): activating background_task = fm_sitrep_gc iteration = 55 reason = Timeout
248142025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
248152025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
248162025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
248172025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
248182025-11-13T08:12:32.648ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
248192025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
248202025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
248212025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
248222025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
248232025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
248242025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
248252025-11-13T08:12:32.649ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12225, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6761, tv_nsec: 1318539 } })
248262025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
248272025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248282025-11-13T08:12:32.649ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
248292025-11-13T08:12:32.650ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
248302025-11-13T08:12:32.651ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248312025-11-13T08:12:32.652ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Fleet result = Ok(())
248322025-11-13T08:12:32.652ZDEBGnexus (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
248332025-11-13T08:12:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
248342025-11-13T08:12:32.653ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
248352025-11-13T08:12:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_watcher resource = Database result = Ok(())
248362025-11-13T08:12:32.653ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
248372025-11-13T08:12:32.654ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
248382025-11-13T08:12:32.654ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248392025-11-13T08:12:32.654ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248402025-11-13T08:12:32.656ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Fleet result = Ok(())
248412025-11-13T08:12:32.656ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_loader roles = RoleSet { roles: {} }
248422025-11-13T08:12:32.656ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_loader resource = Database result = Ok(())
248432025-11-13T08:12:32.659ZDEBGnexus (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6761, tv_nsec: 11127362 } })
248442025-11-13T08:12:32.659ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248452025-11-13T08:12:32.659ZINFOnexus (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1332 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
248462025-11-13T08:12:32.660ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_loader elapsed = 12.22971ms iteration = 55 status = "NoSitrep"
248472025-11-13T08:12:32.663ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
248482025-11-13T08:12:32.663ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
248492025-11-13T08:12:32.663ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
248502025-11-13T08:12:32.691ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
248512025-11-13T08:12:32.691ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 64.807684ms iteration = 82 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
248522025-11-13T08:12:32.711ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
248532025-11-13T08:12:32.712ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
248542025-11-13T08:12:32.714ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248552025-11-13T08:12:32.717ZINFOnexus (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
248562025-11-13T08:12:32.717ZDEBGnexus (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
248572025-11-13T08:12:32.717ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
248582025-11-13T08:12:32.717ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
248592025-11-13T08:12:32.718ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
248602025-11-13T08:12:32.718ZDEBGnexus (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
248612025-11-13T08:12:32.718ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = support_bundle_collector resource = Database result = Ok(())
248622025-11-13T08:12:32.719ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 70.851873ms iteration = 28 status = {"errors":[],"finish_invoked_ok":[]}
248632025-11-13T08:12:32.719ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
248642025-11-13T08:12:32.719ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
248652025-11-13T08:12:32.720ZINFOnexus (BackgroundTasks): No bundles to collect background_task = support_bundle_collector file = nexus/src/app/background/tasks/support_bundle_collector.rs:413
248662025-11-13T08:12:32.720ZDEBGnexus (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 94.269653ms iteration = 42 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
248672025-11-13T08:12:32.723ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248682025-11-13T08:12:32.725ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
248692025-11-13T08:12:32.728ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
248702025-11-13T08:12:32.728ZDEBGnexus (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 79.691616ms iteration = 28 status = {"errors":[],"requests_created_ok":[]}
248712025-11-13T08:12:32.728ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
248722025-11-13T08:12:32.728ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
248732025-11-13T08:12:32.729ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement resource = Database result = Ok(())
248742025-11-13T08:12:32.729ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248752025-11-13T08:12:32.730ZINFOnexus (BackgroundTasks): all instance checks complete background_task = instance_watcher file = nexus/src/app/background/tasks/instance_watcher.rs:552 pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
248762025-11-13T08:12:32.730ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 87.791076ms iteration = 28 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
248772025-11-13T08:12:32.730ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
248782025-11-13T08:12:32.731ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
248792025-11-13T08:12:32.731ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {} }
248802025-11-13T08:12:32.731ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 83.540293ms iteration = 28 status = {"errors":[],"garbage_collect_requested":[]}
248812025-11-13T08:12:32.732ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Database result = Ok(())
248822025-11-13T08:12:32.733ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: 370ec2da-8f7f-4861-9b77-369c2bb1c86d (sled), lookup_type: ById(370ec2da-8f7f-4861-9b77-369c2bb1c86d) } result = Ok(())
248832025-11-13T08:12:32.733ZDEBGnexus (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
248842025-11-13T08:12:32.733ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
248852025-11-13T08:12:32.733ZDEBGnexus (BackgroundTasks): roles background_task = fm_sitrep_gc roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
248862025-11-13T08:12:32.735ZDEBGnexus (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = fm_sitrep_gc resource = Fleet result = Ok(())
248872025-11-13T08:12:32.735ZDEBGnexus (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 92.823345ms iteration = 28 status = 0
248882025-11-13T08:12:32.736ZINFOnexus (BackgroundTasks): sitrep garbage collection found 0 orphaned sitreps, deleted 0 background_task = fm_sitrep_gc file = nexus/src/app/background/tasks/fm_sitrep_gc.rs:112
248892025-11-13T08:12:32.736ZDEBGnexus (BackgroundTasks): activation complete background_task = fm_sitrep_gc elapsed = 87.542281ms iteration = 55 status = {"errors":[],"orphaned_sitreps_deleted":0,"orphaned_sitreps_found":0}
248902025-11-13T08:12:32.736ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
248912025-11-13T08:12:32.737ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
248922025-11-13T08:12:32.740ZDEBGnexus (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 114.043082ms iteration = 45 status = {"physical_disks_added":0}
248932025-11-13T08:12:32.756ZERROnexus (BackgroundTasks): dendrite for switch1 is unavailable, cannot perform nat cleanup background_task = nat_garbage_collector file = nexus/src/app/background/tasks/nat_cleanup.rs:89
248942025-11-13T08:12:32.758ZDEBGnexus (BackgroundTasks): activation complete background_task = nat_garbage_collector elapsed = 151.165952ms iteration = 28 status = {"error":"dendrite for switch1 is unavailable, cannot perform nat cleanup"}
248952025-11-13T08:12:32.770ZINFOnexus (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
248962025-11-13T08:12:32.770ZDEBGnexus (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
248972025-11-13T08:12:32.770ZDEBGnexus (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
248982025-11-13T08:12:32.771ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = instance_updater resource = Database result = Ok(())
248992025-11-13T08:12:32.780ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement elapsed = 137.331791ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
249002025-11-13T08:12:32.781ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 138.424791ms iteration = 82 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
249012025-11-13T08:12:32.783ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1340 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 zone_address = fd00:1122:3344:101::2
249022025-11-13T08:12:32.783ZDEBGnexus (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
249032025-11-13T08:12:32.784ZDEBGnexus (MgsClient): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "adafc419-f489-415e-b2d5-23a5871977d5", "content-length": "26", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249042025-11-13T08:12:32.784ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1343 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
249052025-11-13T08:12:32.785ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = switch_port_config_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n} rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
249062025-11-13T08:12:32.785ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 136.994994ms iteration = 28 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
249072025-11-13T08:12:32.791ZDEBGnexus (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
249082025-11-13T08:12:32.791ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
249092025-11-13T08:12:32.800ZINFOnexus (BackgroundTasks): determining switch slot managed by switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1340 zone_address = fd00:1122:3344:101::2
249102025-11-13T08:12:32.800ZDEBGnexus (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/switch-id
249112025-11-13T08:12:32.800ZDEBGnexus (MgsClient): client response background_task = bfd_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/switch-id", status: 200, headers: {"content-type": "application/json", "x-request-id": "61a09cd6-6c99-4209-a851-e615262662fa", "content-length": "26", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249122025-11-13T08:12:32.800ZINFOnexus (BackgroundTasks): identified switch slot for switch zone background_task = bfd_manager file = nexus/src/app/mod.rs:1343 slot = SpIdentifier {\n slot: 0,\n type_: Switch,\n} zone_address = fd00:1122:3344:101::2
249132025-11-13T08:12:32.802ZINFOnexus (BackgroundTasks): completed mapping switch zones to switch slots background_task = bfd_manager file = nexus/src/app/mod.rs:1378 mappings = {\n Switch0: fd00:1122:3344:101::2,\n}
249142025-11-13T08:12:32.819ZINFOnexus (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0 file = nexus/src/app/background/tasks/instance_updater.rs:59
249152025-11-13T08:12:32.819ZINFOnexus (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 file = nexus/src/app/background/tasks/instance_updater.rs:225 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
249162025-11-13T08:12:32.819ZDEBGnexus (BackgroundTasks): activation complete background_task = instance_updater elapsed = 175.660224ms iteration = 28 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
249172025-11-13T08:12:32.830ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
249182025-11-13T08:12:32.831ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
249192025-11-13T08:12:32.832ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = GET uri = http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids
249202025-11-13T08:12:32.833ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/local/all-sp-ids", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecca9007-b928-4910-8529-a28ed88af49d", "content-length": "78", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249212025-11-13T08:12:32.833ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
249222025-11-13T08:12:32.833ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
249232025-11-13T08:12:32.833ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
249242025-11-13T08:12:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
249252025-11-13T08:12:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
249262025-11-13T08:12:32.833ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
249272025-11-13T08:12:32.835ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = switch
249282025-11-13T08:12:32.835ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 0 sp_type = sled
249292025-11-13T08:12:32.837ZDEBGnexus (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 189.555996ms iteration = 28 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
249302025-11-13T08:12:32.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = sled
249312025-11-13T08:12:32.838ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 0 sp_type = switch
249322025-11-13T08:12:32.838ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = switch
249332025-11-13T08:12:32.838ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 0 sp_type = sled
249342025-11-13T08:12:32.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = sled
249352025-11-13T08:12:32.838ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 0 sp_type = switch
249362025-11-13T08:12:32.840ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 9ced08ad-49c8-4784-b628-30a17002ba1f (ereporter_restart) slot = 0 sp_type = sled start_ena = Some(Ena(0x2))
249372025-11-13T08:12:32.840ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2
249382025-11-13T08:12:32.840ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/0/ereports?committed=1&limit=255&restart_id=9ced08ad-49c8-4784-b628-30a17002ba1f&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "62e00265-d799-4856-8682-df5a0fc204c7", "content-length": "81", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249392025-11-13T08:12:32.873ZDEBGnexus (DnsResolver): lookup_all_socket_v6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: SRV(SRV { priority: 0, weight: 0, port: 12224, target: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal.") }) }, Record { name_labels: Name("dendrite-370ec2da-8f7f-4861-9b77-369c2bb1c86d.host.control-plane.oxide.internal."), dns_class: IN, ttl: 0, rdata: AAAA(AAAA(fd00:1122:3344:101::2)) }], valid_until: Instant { tv_sec: 6761, tv_nsec: 225960526 } })
249402025-11-13T08:12:32.874ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:98
249412025-11-13T08:12:32.874ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
249422025-11-13T08:12:32.874ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "518e575c-327a-4b16-8a37-7be74fbe1b14", "content-length": "372", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249432025-11-13T08:12:32.874ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249442025-11-13T08:12:32.875ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249452025-11-13T08:12:32.876ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} } slot = 1 sp_type = sled
249462025-11-13T08:12:32.876ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 8f476d05-652f-489b-b66c-4f69247a3bb4 (ereporter_restart) slot = 0 sp_type = switch start_ena = Some(Ena(0x2))
249472025-11-13T08:12:32.876ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2
249482025-11-13T08:12:32.876ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/switch/0/ereports?committed=1&limit=255&restart_id=8f476d05-652f-489b-b66c-4f69247a3bb4&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "6df8211e-088f-4c7d-b02d-61bfa9ca2bc4", "content-length": "81", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249492025-11-13T08:12:32.878ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Fleet result = Ok(()) slot = 1 sp_type = sled
249502025-11-13T08:12:32.878ZDEBGnexus (BackgroundTasks): roles background_task = sp_ereport_ingester roles = RoleSet { roles: {} } slot = 1 sp_type = sled
249512025-11-13T08:12:32.878ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = sp_ereport_ingester resource = Database result = Ok(()) slot = 1 sp_type = sled
249522025-11-13T08:12:32.880ZDEBGnexus (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 273.126309ms iteration = 28 status = {}
249532025-11-13T08:12:32.930ZDEBGnexus (BackgroundTasks): attempting ereport collection from MGS background_task = sp_ereport_ingester committed_ena = Some(Ena(0x1)) gateway_addr = [fd00:1122:3344:101::2]:12225 restart_id = 68307b96-5403-4fcc-b137-d1c2720d7b08 (ereporter_restart) slot = 1 sp_type = sled start_ena = Some(Ena(0x2))
249542025-11-13T08:12:32.930ZDEBGnexus (BackgroundTasks): client request background_task = sp_ereport_ingester body = None gateway_url = http://[fd00:1122:3344:101::2]:12225 method = POST uri = http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2
249552025-11-13T08:12:32.931ZDEBGnexus (BackgroundTasks): client response background_task = sp_ereport_ingester gateway_url = http://[fd00:1122:3344:101::2]:12225 result = Ok(Response { url: "http://[fd00:1122:3344:101::2]:12225/sp/sled/1/ereports?committed=1&limit=255&restart_id=68307b96-5403-4fcc-b137-d1c2720d7b08&start_at=2", status: 200, headers: {"content-type": "application/json", "x-request-id": "be636d6b-371a-476b-88d9-a689732ab56b", "content-length": "81", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249562025-11-13T08:12:32.931ZINFOnexus (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:116 sled = sock
249572025-11-13T08:12:32.931ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/eip-gateways
249582025-11-13T08:12:32.932ZDEBGnexus (BackgroundTasks): activation complete background_task = sp_ereport_ingester elapsed = 284.047314ms iteration = 28 status = {"disabled":false,"errors":[],"sps":[]}
249592025-11-13T08:12:32.932ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/eip-gateways", status: 204, headers: {"x-request-id": "79aa8f3d-638c-44e9-83a7-b2df864ffb04", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249602025-11-13T08:12:32.932ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249612025-11-13T08:12:32.933ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249622025-11-13T08:12:32.936ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249632025-11-13T08:12:32.936ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249642025-11-13T08:12:32.939ZDEBGnexus (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
249652025-11-13T08:12:32.940ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = vpc_route_manager resource = Database result = Ok(())
249662025-11-13T08:12:32.942ZINFOnexus (BackgroundTasks): VPC route manager sled 370ec2da-8f7f-4861-9b77-369c2bb1c86d push not needed background_task = vpc_route_manager file = nexus/src/app/background/tasks/vpc_routes.rs:262
249672025-11-13T08:12:32.942ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[fd00:1122:3344:101::1]:12345/vpc-routes
249682025-11-13T08:12:32.943ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = vpc_route_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/vpc-routes", status: 204, headers: {"x-request-id": "44edbf18-778f-479f-ad17-bbf48f29a7cb", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
249692025-11-13T08:12:32.945ZDEBGnexus (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 302.403236ms iteration = 42 status = {}
249702025-11-13T08:12:32.947ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
249712025-11-13T08:12:32.947ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
249722025-11-13T08:12:32.950ZINFOnexus (BackgroundTasks): fetching switch port settings background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:150 port = SwitchPort { id: 26f97b2b-bb8a-464f-8c85-1ca099fbb449, rack_id: 49e80ef0-2c44-4164-bdad-87e8ec30f2e5, switch_location: "switch0", port_name: Name(Name("qsfp0")), port_settings_id: Some(9992099f-a984-400e-9195-42b03b61d9a7) } rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
249732025-11-13T08:12:32.950ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
249742025-11-13T08:12:32.950ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
249752025-11-13T08:12:32.951ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
249762025-11-13T08:12:32.952ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
249772025-11-13T08:12:32.953ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
249782025-11-13T08:12:32.956ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
249792025-11-13T08:12:32.956ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
249802025-11-13T08:12:32.956ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
249812025-11-13T08:12:32.957ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
249822025-11-13T08:12:32.960ZDEBGnexus (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Fleet result = Ok(())
249832025-11-13T08:12:32.960ZDEBGnexus (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
249842025-11-13T08:12:32.960ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = v2p_manager resource = Database result = Ok(())
249852025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): retrieved port settings from switch background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1842 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 settings = PortSettings { links: {"0": LinkSettings { addrs: [10.151.2.174], params: LinkCreate { autoneg: false, fec: Some(None), kr: false, lane: Some(LinkId(0)), speed: Speed40G, tx_eq: None } }} } switch_port_id = Name(Name("qsfp0"))
249862025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): port settings up to date, skipping background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:1869 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0 switch_port_id = Name(Name("qsfp0")) switch_port_settings_id = 9992099f-a984-400e-9195-42b03b61d9a7
249872025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): retrieved existing routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:388 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, Some(1))}}
249882025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): retrieved desired routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:392 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: {(10.151.2.100, Prefix4 { length: 0, value: 0.0.0.0 }, None, None)}}
249892025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): calculated static routes to add background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:401 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
249902025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): calculated static routes to delete background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:407 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
249912025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): deleting static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:412 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
249922025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): removing static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2016 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = DeleteStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
249932025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): adding static routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:418 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 routes = {Switch0: AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } }}
249942025-11-13T08:12:32.963ZINFOnexus (BackgroundTasks): adding static v4 routes background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:2052 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 request = AddStaticRoute4Request { routes: StaticRoute4List { list: [StaticRoute4 { nexthop: 10.151.2.100, prefix: Prefix4 { length: 0, value: 0.0.0.0 }, rib_priority: 1, vlan_id: None }] } } switch_location = Switch0
249952025-11-13T08:12:32.964ZINFOnexus (BackgroundTasks): checking for changes to loopback addresses background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:427 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5
249962025-11-13T08:12:32.964ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
249972025-11-13T08:12:32.964ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
249982025-11-13T08:12:32.966ZINFOnexus (BackgroundTasks): applying SMF config uplink updates to switch zone background_task = switch_port_config_manager config = [HostPortConfig { addrs: [UplinkAddressConfig { address: V4(Ipv4Net { addr: 10.151.2.174, width: 24 }), vlan_id: None }], lldp: Some(LldpPortConfig { chassis_id: None, management_addrs: None, port_description: None, port_id: None, status: Disabled, system_description: None, system_name: None }), port: "qsfp0", tx_eq: None }] file = nexus/src/app/background/tasks/sync_switch_configuration.rs:476 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch_location = Switch0
249992025-11-13T08:12:32.966ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = Some(Body) method = POST rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/switch-ports
250002025-11-13T08:12:32.994ZDEBGnexus (BackgroundTasks): client request SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager body = None method = GET uri = http://[fd00:1122:3344:101::1]:12345/v2p
250012025-11-13T08:12:32.995ZDEBGnexus (BackgroundTasks): client response SledAgent = 370ec2da-8f7f-4861-9b77-369c2bb1c86d background_task = v2p_manager result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe196e99-b0ee-4f6c-aba0-050664374a91", "content-length": "691", "date": "Thu, 13 Nov 2025 08:12:32 GMT"} })
250022025-11-13T08:12:32.995ZINFOnexus (BackgroundTasks): found opte v2p mappings background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:118 interfaces = {} sled = sock
250032025-11-13T08:12:32.995ZINFOnexus (BackgroundTasks): v2p mappings to delete background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:130 mappings = [] sled = sock
250042025-11-13T08:12:32.995ZINFOnexus (BackgroundTasks): v2p mappings to add background_task = v2p_manager file = nexus/src/app/background/tasks/v2p_mappings.rs:143 mappings = [] sled = sock
250052025-11-13T08:12:32.997ZDEBGnexus (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 354.892487ms iteration = 28 status = {}
250062025-11-13T08:12:33.126ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/switch-ports", status: 204, headers: {"x-request-id": "f0873ba1-25f8-4e0e-9608-fe065d6f2820", "date": "Thu, 13 Nov 2025 08:12:33 GMT"} })
250072025-11-13T08:12:33.126ZINFOnexus (BackgroundTasks): no bgp config found for switch, skipping. background_task = switch_port_config_manager file = nexus/src/app/background/tasks/sync_switch_configuration.rs:779 rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 switch = Switch0
250082025-11-13T08:12:33.126ZDEBGnexus (BackgroundTasks): client request background_task = switch_port_config_manager body = None method = GET rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 uri = http://[fd00:1122:3344:101::1]:12345/network-bootstore-config
250092025-11-13T08:12:33.133ZDEBGnexus (BackgroundTasks): client response background_task = switch_port_config_manager rack_id = 49e80ef0-2c44-4164-bdad-87e8ec30f2e5 result = Ok(Response { url: "http://[fd00:1122:3344:101::1]:12345/network-bootstore-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5d4573c-5a19-4e38-a32f-4b732c4ad2a1", "content-length": "682", "date": "Thu, 13 Nov 2025 08:12:33 GMT"} })
250102025-11-13T08:12:33.133ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250112025-11-13T08:12:33.133ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
250122025-11-13T08:12:33.136ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250132025-11-13T08:12:33.137ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
250142025-11-13T08:12:33.139ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250152025-11-13T08:12:33.139ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
250162025-11-13T08:12:33.141ZDEBGnexus (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
250172025-11-13T08:12:33.141ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
250182025-11-13T08:12:33.148ZDEBGnexus (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 506.471807ms iteration = 28 status = {}
250192025-11-13T08:12:33.747ZDEBGnexus (producer-registration-task): registering as oximeter producer with Nexus registration_count = 5
250202025-11-13T08:12:33.747ZDEBGnexus (producer-registration-task): will register with Nexus at [fd00:1122:3344:101::a]:12221
250212025-11-13T08:12:33.778ZDEBGnexus (producer-registration-task): client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/producers
250222025-11-13T08:12:33.778ZINFOnexus (dropshot_internal): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:1025 local_addr = [fd00:1122:3344:101::a]:12221 remote_addr = [fd00:1122:3344:101::a]:45955
250232025-11-13T08:12:33.778ZDEBGnexus (dropshot_internal): determined request API version local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:45955 req_id = 9b4c4747-50ab-457d-924d-09d0a3af74f3 uri = /metrics/producers version = 1.0.0
250242025-11-13T08:12:33.778ZDEBGnexus (dropshot_internal): roles authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:45955 req_id = 9b4c4747-50ab-457d-924d-09d0a3af74f3 roles = RoleSet { roles: {} } type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
250252025-11-13T08:12:33.778ZDEBGnexus (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) authenticated = true local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:45955 req_id = 9b4c4747-50ab-457d-924d-09d0a3af74f3 resource = Database result = Ok(()) type = user_builtin uri = /metrics/producers user_builtin_id = 001de000-05e4-4000-8000-000000000002
250262025-11-13T08:12:33.817ZINFOnexus: assigned collector to new producer collector_id = 87d2803d-0c3b-4aec-a08d-4edc5d8a6a5d file = nexus/src/app/oximeter.rs:76 producer_id = bcb58129-79cc-406c-aa16-455a5cd9f031
250272025-11-13T08:12:33.817ZINFOnexus (dropshot_internal): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 39397 local_addr = [fd00:1122:3344:101::a]:12221 method = POST remote_addr = [fd00:1122:3344:101::a]:45955 req_id = 9b4c4747-50ab-457d-924d-09d0a3af74f3 response_code = 201 uri = /metrics/producers
250282025-11-13T08:12:33.817ZDEBGnexus (producer-registration-task): client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "9b4c4747-50ab-457d-924d-09d0a3af74f3", "content-length": "41", "date": "Thu, 13 Nov 2025 08:12:33 GMT"} })
250292025-11-13T08:12:33.818ZDEBGnexus (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
250302025-11-13T08:12:35.481ZINFOnexus (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.4/src/server.rs:867 latency_us = 506 local_addr = [fd00:1122:3344:101::a]:32941 method = GET remote_addr = [fd00:1122:3344:101::d]:52304 req_id = 8f5e1050-5845-4cda-b024-5a1b8e88f443 response_code = 200 uri = /bcb58129-79cc-406c-aa16-455a5cd9f031
250312025-11-13T08:12:37.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 164 reason = Timeout
250322025-11-13T08:12:37.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
250332025-11-13T08:12:37.626ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
250342025-11-13T08:12:37.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250352025-11-13T08:12:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
250362025-11-13T08:12:37.631ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
250372025-11-13T08:12:37.631ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
250382025-11-13T08:12:37.633ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
250392025-11-13T08:12:37.633ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 6.997698ms iteration = 164 status = {"config_updated":false}
250402025-11-13T08:12:42.626ZDEBGnexus (BackgroundTasks): activating background_task = blueprint_loader iteration = 83 reason = Timeout
250412025-11-13T08:12:42.626ZDEBGnexus (BackgroundTasks): activating background_task = reconfigurator_config_watcher iteration = 165 reason = Timeout
250422025-11-13T08:12:42.626ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
250432025-11-13T08:12:42.626ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
250442025-11-13T08:12:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
250452025-11-13T08:12:42.627ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
250462025-11-13T08:12:42.628ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250472025-11-13T08:12:42.628ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250482025-11-13T08:12:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
250492025-11-13T08:12:42.632ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
250502025-11-13T08:12:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
250512025-11-13T08:12:42.632ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = BlueprintConfig result = Ok(())
250522025-11-13T08:12:42.632ZDEBGnexus (BackgroundTasks): roles background_task = reconfigurator_config_watcher roles = RoleSet { roles: {} }
250532025-11-13T08:12:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = reconfigurator_config_watcher resource = Database result = Ok(())
250542025-11-13T08:12:42.633ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
250552025-11-13T08:12:42.633ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
250562025-11-13T08:12:42.634ZDEBGnexus (BackgroundTasks): reconfigurator config load complete background_task = reconfigurator_config_watcher switches = Loaded(ReconfiguratorConfigView { version: 1, config: ReconfiguratorConfig { planner_enabled: true, planner_config: PlannerConfig { add_zones_with_mupdate_override: false } }, time_modified: 1970-01-01T00:01:01Z })
250572025-11-13T08:12:42.634ZDEBGnexus (BackgroundTasks): activation complete background_task = reconfigurator_config_watcher elapsed = 8.004177ms iteration = 165 status = {"config_updated":false}
250582025-11-13T08:12:42.635ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
250592025-11-13T08:12:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200, lookup_type: ById(9f9c0cba-5148-41b6-9e38-c1bc1bf5a200) } result = Ok(())
250602025-11-13T08:12:42.638ZDEBGnexus (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
250612025-11-13T08:12:42.638ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = blueprint_loader resource = Database result = Ok(())
250622025-11-13T08:12:42.642ZDEBGnexus (BackgroundTasks): activating background_task = region_replacement_driver iteration = 83 reason = Timeout
250632025-11-13T08:12:42.642ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
250642025-11-13T08:12:42.642ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
250652025-11-13T08:12:42.653ZDEBGnexus (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 original_time_created = 2025-11-13 07:59:12.398172 UTC target_id = 9f9c0cba-5148-41b6-9e38-c1bc1bf5a200 time_created = 2025-11-13 07:59:12.398172 UTC
250662025-11-13T08:12:42.654ZDEBGnexus (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 27.295386ms iteration = 83 status = {"enabled":true,"status":"target blueprint unchanged","target_id":"9f9c0cba-5148-41b6-9e38-c1bc1bf5a200","time_created":"2025-11-13T07:59:12.398172Z"}
250672025-11-13T08:12:42.677ZDEBGnexus (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
250682025-11-13T08:12:42.677ZDEBGnexus (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002 (built_in_user), .. }) background_task = region_replacement_driver resource = Database result = Ok(())
250692025-11-13T08:12:42.711ZDEBGnexus (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 68.619399ms iteration = 83 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}